Skip to main content
Download PDF

Internal validation and evaluation of the predictive performance of models based on the PRISM-3 (Pediatric Risk of Mortality) and PIM-3 (Pediatric Index of Mortality) scoring systems for predicting mortality in Pediatric Intensive Care Units (PICUs)

BMC Pediatrics volume 22, Article number: 199 (2022) Cite this article

Abstract

Purpose

The study was aimed to assess the prognostic power The Pediatric Risk of Mortality-3 (PRISM-3) and the Pediatric Index of Mortality-3 (PIM-3) to predict in-hospital mortality in a sample of patients admitted to the PICUs.

Design and methods

The study was performed to include all children younger than 18 years of age admitted to receive critical care in two hospitals, Mashhad, northeast of Iran from December 2017 to November 2018. The predictive performance was quantified in terms of the overall performance by measuring the Brier Score (BS) and standardized mortality ratio (SMR), discrimination by assessing the AUC, and calibration by applying the Hosmer-Lemeshow test.

Results

A total of 2446 patients with the median age of 4.2 months (56% male) were included in the study. The PICU and in-hospital mortality were 12.4 and 16.14%, respectively. The BS of the PRISM-3 and PIM-3 was 0.088 and 0.093 for PICU mortality and 0.108 and 0.113 for in-hospital mortality. For the entire sample, the SMR of the PRISM-3 and PIM-3 were 1.34 and 1.37 for PICU mortality and 1.73 and 1.78 for in-hospital mortality, respectively. The PRISM-3 demonstrated significantly higher discrimination power in comparison with the PIM-3 (AUC = 0.829 vs 0.745) for in-hospital mortality. (AUC = 0.779 vs 0.739) for in-hospital mortality. The HL test revealed poor calibration for both models in both outcomes.

Conclusions

The performance measures of PRISM-3 were better than PIM-3 in both PICU and in-hospital mortality. However, further recalibration and modification studies are required to improve the predictive power to a clinically acceptable level before daily clinical use.

Practice implications

The calibration of the PRISM-3 model is more satisfactory than PIM-3, however both models have fair discrimination power.

Peer Review reports

Introduction

Recent advances in therapeutic protocols and medical facilities highlight the need for accurate prediction systems [1]. Such risk prediction models can be used in tasks such as benchmarking for the evaluation the effectiveness and efficiency of pediatric intensive care (PICUs), early detection of critically ill patients, and optimizing resource allocation which may result in better quality of care and patient safety, especially in low and middle-income countries [2]. These countries including Iran have scarce resources, especially when a surge of critically ill pediatric patients leads to disproportionate disbalance between needs and available resources.

In this circumstance, employing an accurate, well-validated, and easy-to-calculate risk assessment instrument can benefit prioritizing patients and optimizing resource use in the PICUs [3, 4]. Such risk assessment instruments can be based on scoring systems, which use the worst physiological and laboratory values during the first 12–24 h of admission to indicate severity of illness. A higher score represents higher severity [5].

Multiple scoring systems have been introduced, some of which are widely used to predict the risk of death in children such as the Pediatric Risk of Mortality (PRISM) and the Pediatric Index of Mortality (PIM). The PRISM was developed using the data collected from 11,165 patients admitted to PICUs in the USA [6] whereas, the PIM was developed based on the data of ICUs located in the UK, Australia, Ireland, and New Zealand [7]. The third version of these scoring systems (PRISM-3 and PIM-3) is commonly used in the Intensive Care Units (ICUs) for years after its introduction [8]. Over the last decade, there have been significant advances in pediatric intensive care among developing countries. However, in countries with low- and middle-income as well as with a higher pediatric population, there is still a need to PICUs, a greater number of competent health care professionals, timely access to required medicine, and equipment to successfully contribute to the reduction of pediatric mortality. The predictive performance of models based on the PRISM-3 and PIM-3 scores for PICU mortality and in-hospital mortality are not well understood, especially in developing countries including our country. Hence, this study is aimed at evaluating and comparing the predictive performance of prediction models based on the PRISM-3 and PIM-3 scores in a sample of patients admitted to the PICU a developing country [9]. Hence, this study is aimed at evaluating and comparing the predictive performance of prediction models based on the PRISM-3 and PIM-3 scores in a sample of patients admitted to the PICU a developing country.

Method

Study design and setting

We designed a multicenter, retrospective cohort study of severely ill children admitting to the six tertiary PICUs at two university hospitals for a period of 12 months, from December 2017 to November 2018, in Mashhad, northeast of Iran. Each hospital had an average of 1500 admissions per year and each PICU was equipped with an average of 7.2 beds.

Both centers are general pediatric hospitals and admit all sorts of cases (medical and surgical). However according to the subspecialty approach of the hospitals, the majority of oncology, nephrology and hematology cases were treated in “hospital A” and many cases of surgical, rheumatology, lung, infectious, gastrointestinal and neurological were treated to “hospital B”. The PICUs in each hospital do not differ in terms of the type of patients referred, but since smaller beds are intended for younger patients, the only difference is related to age classification.

Study population

All children (i.e., aged younger than 18 years) admitted to the PICU were eligible in the study. We excluded from the analysis patients with brain death at the time of admission, and patients who stayed in PICU for less than 2 h and discharge or expire before 24 h of admission. In addition, those patients who were referred to subspecialty hospitals are also excluded. Individuals with missing values for the main variables which is essential for calculating the scores were imputed using the chained equations approach implemented in the mice package available in R. It should be noted that re-admissions due to different diagnoses were considered as new admissions.

Study variables

The following variables were collected: age, gender, diagnosis, other main variables for calculating the scores, as well as length of stay (LOS) at both PICU and hospital, and the two outcome variables PICU mortality and in-hospital mortality.

The key variables were collected to calculate the PRISM-3 score: arterial blood gas, glucose, creatinine, Glasgow Coma Score (GCS), respiratory rate, systolic and diastolic blood pressure, heart rate, pupillary reactions to bright light, blood urea, potassium, platelet, white blood cell count temperature, Prothrombin Time (PT), and Partial Thromboplastin Time (PTT) [5]. The variables used to calculate the PIM-3 score were as follows: Low, high, or very high-risk diagnosis. Low-risk diagnosis including: asthma, bronchiolitis, croup, obstructive sleep apnea, diabetic, ketoacidosis, and seizure. High-risk diagnosis including: spontaneous cerebral hemorrhage, cardiomyopathy or myocarditis, hypoplastic left heart syndrome, neurodegenerative disorder, necrotizing enterocolitis. Very high-risk diagnosis including: cardiac arrest, severe combined immune deficiency, leukemia or lymphoma, bone marrow transplant recipient, and liver failure.

systolic blood pressure, base excess, type of admission (Emergency, referral, and elective), FiO2, PaO2, mechanical ventilation support, recovery from surgery as the main reason for admission to the PICU, and admission due to cardiac bypass [10]. The other vital signs and.

Since these two models categorized the age variable according to the month. We also considered the month as a unit of age (see Table 1 which summarizes these models in terms of variables, unit of variables, the formula for calculating and their point assignment schemes). It should be noted that all of the units associated with each variable is provided in Tables 2 and 3. The unit for the LOS was consider day.

Table 1 The point assignment scheme of each scoring system
Full size table
Table 2 Baseline demographic and clinical characteristics of the patients admitted to the PICU with available and missing data
Full size table
Table 3 Baseline demographic and clinical characteristics of the patients admitted to the PICU after imputation
Full size table

Statistical analyses

Normality of continuous variables was assessed via the ShapiroWilk test. The data did not follow a normal distribution, so we compared the groups by utilizing nonparametric techniques. The Mann-Whitney U test was used for comparisons of continuous variables between survivors and non-survivors. The Chi-square test or Fisher’s exact test were also used to compare categorical data. Data were presented as median (IQR) for continuous variables and as the frequency (%) for categorical variables.

The PIM-3 and PRISM-3 sores were calculated retrospectively by the researchers for each patient based on the measurements at the time of admission to the PICU. The formula for calculating PRISM-3 and PIM-3 score is presented in Table 1.

After calculating the scores of each scoring system, we applied logistic regression analysis to predict both PICU and in-hospital mortality as response variable by using the PRISM-3 and PIM-3 scores as the explanatory variables, separately. The logit formula was used to calculate the probability of mortality as following:

$$\mathrm{P}=\frac{1}{1+\exp \left[-\left({\beta}_0+{\beta}_1X\right)\right]}$$

(β0: Intercept; β1: Coefficient of the score; X: score)

Then the predictive performance of the models was assessed in terms of the overall accuracy, discrimination, and calibration. The discrimination ability of the probabilistic models as measured by the AUC is exactly the same as the discrimination ability of the original scores they are based on. This is because the model keeps the same order in the probabilities as in the scores (i.e. if we sort the probabilities in ascending order it will result in the same order as with the score). The probabilistic model however allows us to investigate the additional performance measures of calibration and the Brier score.

The predictive performance of the models was quantified with respect to the accuracy of the predicted probabilities, discrimination, and calibration. The accuracy between the predicted and observed probabilities was assessed by the Brier Score (BS), which is the mean squared difference between the observed and predicted outcome and using a standardized mortality ratio (SMR), which is the ratio of the risk-adjusted observed mortality to the expected mortality derived from the development set where the score was developed. Discrimination between survivors and non-survivors was quantified by the Area Under the Receiver Operating Characteristic Curve (AUC). Calibration, which is a measure of the agreement between the predicted and observed probabilities was assessed by calibration and lack of agreement was tested by the Hosmer-Lemshow. Moreover, the Negative Predictive Value (NPV), Positive Predictive Value (PPV), specificity, and sensitivity were calculated using the Youden Index threshold [11]. We used bootstrapping with 1000 samples to internally validate the model and calculate the bias-corrected estimate of the AUC and its confidence intervals (CI) and the Delong’s method was used to compare the two AUCs. Statistical significance was set at the 0.05 p-value level. All analyses were performed using the R statistical environment (with packages rms, Hmisc, pROC, and mice).

Results

In total, 3000 patients were eligible and met the inclusion criteria. After applying the exclusion criteria, 2784 patients remained for further analyses (Fig. 1). The data had about 11.3% of missing values, which were imputed as described in the Statistical Analysis section.

Fig. 1
figure 1

The flowchart diagram of the patient inclusion process

Full size image

The PICU and in-hospital mortality were 12.14 and 15.58%, respectively. Table 2 and Table 3 demonstrate the baseline characteristics of the study population before and after imputation. The median length of both the PICU and hospital stay were 7(3–13) and 8(4.7–16), respectively (See Table 2 and Table 3).

A total of 1379 (56.4%) patients were male and the median age of the patients was 4.2 months (IQR: 0.66–24) the majority of the patients were younger than 12 months (65.43%). Generally, of the demographic profile, age was associated with outcome (p < 0.001) while gender did not show any significant influence on the outcome (p = 0.13).

The congenital malformation, digestive system disease, and patient with the respiratory diseases accounted for 24.5, 13.8, and 10.6% of the admissions, respectively. The cause of mortality according to the ICD-10 coding system were as follows: 88 (22.3%) congenital malformations, 54 (13.7%) digestive system diseases, 51 (12.9%) the respiratory diseases, 43 (10. 9%) blood and neoplasm diseases, 23 (5.82%) kidney diseases, 22 (5.6%) infectious diseases, 18 (4.6%) metabolic diseases, 14 (3.5%) cardiac disease, 8 (2%) neurological disorders, 7 (1.8%) perinatal diseases, and 67 (16.9%) other diagnostic groups.

The mean score of PRISM-3 and PIM-3 were 6.9 ± 6.5 and 3 ± 2.8 respectively. About 48% of patient were referral cases, 30% were brought in by emergency medical services, and 9.4% of patients required mechanical ventilation support.

As shown in Tables 2, 3 and Table 4, predominantly those patients in the age group of 12–144 months had the worst outcome, and this pattern is similar in both PICU (19%) and in-hospital mortality (38.5%). These patients were mainly assigned the diagnosis belonging to neoplasm, circulatory, respiratory, and also digestive system categories.

Table 4 Predictive characteristics of PRISM-3 and PIM-3 to predict PICU and in-hospital mortality
Full size table

The linear predictors of the logistic regression models presented per outcome, separately, are:

For predicting PICU mortality:

$$\mathrm{PRISM}-3:-3.056+0.174\times \mathrm{PRISM}-3\_\mathrm{score},\mathrm{and}\ \mathrm{PIM}-3:-3.075+0.297\times \mathrm{PIM}-3\_\mathrm{score}.$$

For predicting in-hospital mortality:

$$\mathrm{PRISM}-3:-3.094+0.166\times \mathrm{PRISM}-3\_\mathrm{score},\mathrm{and}\ \mathrm{PIM}-3:-2.772+0.312\times \mathrm{PIM}-3\_\mathrm{score}.$$

The BS, SMR, AUC, HL-test, and other characteristics of both models for PICU and in-hospital mortality prediction, as well as according to age groups are presented in Table 4. The BS of the PRISM-3 and PIM-3 was 0.088 and 0.093 for PICU mortality and 0.108 and 0.113 for in-hospital mortality. The SMR of the PRISM-3 and PIM-3 was 1.34 (CI 95%: 1.19–1.49) and 1.37 (CI 95%: 1.21–1.52) for PICU mortality and 1.73 (CI 95%: 1.56–1.90) and 1.78 (CI 95%: 1.6–1.95) for in-hospital mortality, respectively. The PRISM-3 demonstrated significantly higher discrimination power in comparison with the PIM-3 (AUC = 0.831 vs 0.745) for in-hospital mortality and (AUC = 0.781 vs 0.737) for in-hospital mortality. The HL test revealed poor calibration for both models in both outcomes. The difference in the AUCs for PRISM-3 and PIM-3 models are significantly significant (P = 0.001) (see Fig. 2 and Table 4). The calibration graphs of both models are shown in Fig. 3.

Fig. 2
figure 2

Receiver operating characteristic curve of the PRISM-3 and PIM-3 in hospitals

Full size image
Fig. 3
figure 3

Calibration curves for the observed mortality against predicted risk of death for PIM-3 and PRISM-3 models in hospitals

Full size image

Discussion

Main findings

This multi-center study aimed to evaluate the performance of the models based on the PRISM-3 and PIM-3 scores in predicting both PICU and in-hospital mortality. We found that the overall performance of PRISM-3 and PIM-3 were comparable for in-hospital mortality in terms of the Brier score. The discrimination power of PRISM-3, however, was significantly higher than the PIM-3 for both PICU and in-hospital mortality. Interestingly, when considering PICU mortality as an outcome, the PRISM-3 appears to be much more discriminative (AUC: 0.78 vs 0.83). A possible explanation is that predicting a short-term outcome is easier than a longer-term outcome. With the exception of the adolescent age group, the PRISM-3 was far superior in predicting PICU and hospital mortality than PIM-3.

The models were not well calibrated in predicting PICU mortality nor in-hospital morality. One possible explanation is that the original models were developed for western populations and are now being applied to an Asian population in a developing country. Generally, with respect to the discrimination ability, the PRISM-3 performed significantly better than the PIM-3. A possible explanation for this is the consideration of more important factors. However, PRISM-3 requires the collection of 17 variables while the PIM-3 requires the collection of only 12 variables which makes the former a more demanding model [11]. Practicality, just as clinical sensibility, may play an important role in clinical applications. Generally, the purpose of designing a prediction model is to offer a reliable model that can be transported and used in clinical practice; hence, it is critical to choose a model that is reasonably simple but does not sacrifice substantial predictive performance. The requirement for a succinct decision method may be even more important in the PICUs in developing countries, where clinicians frequently deal with complicated and severely ill children as well as limited resources. Having an objective method, using either the more complex model if applicable, or the simpler model if that is opportune, can assist them in prioritizing and managing complex patients, as well as enhancing benchmarking indices. Our findings reveal that the PICU and in-hospital mortality were 12.4 and 16.4%, respectively. The PICU mortality in our study is much higher than in European and the US PICUs (12.4 vs 2.5) [12, 13]. However, the PICU mortality rate in our study is situated in the middle of the mortality range in developing countries (range from 8.4% for Korea to 40% for Egypt). There are various reasons for the discrepancy in mortality rate between our study and in western countries. To begin with, the two centers in our study are referral hospitals so they frequently deal with the most critically ill patients. In addition, the disease profile in the present study is also different from studies in western countries. For instance, the majority of patients have also suffered from congenital malformation, digestive and respiratory and cancer disease, and treatments were more challenging for these patients. Furthermore, the other explanation of higher mortality in our study compared to developed countries is the difference in quality and standards of care, equipment used, and the relatively undeveloped medical care level. So, these differences necessitate a significant effort for improvement.

In comparison to previous investigations that have been conducted in Iran, the Middle East, and Asia, this is one of the largest studies that examine the prognostic performance of the PRISM-3 and PIM-3 in predicting pediatric patient outcomes (both PICU and in-hospital mortality). All of those studies were performed in Iran were single center and the median sample size was only 221 (min-max: 90–365) tend to be located at higher frequencies in male gender and the majority of the patients were included samples was younger than 40 months. These investigations determined that the PRISM-3 differential power was between the range of fair (AUC:0.70–0.80) to adequate (AUC:0.80–0.90).

In general, with respect to Table 5, most of related studies have been performed on small samples, the median AUCs for the PIM-3 and PRISM-3 in similar studies were 0.82 [min-max: 0.72–0.89] and 0.82 [min-max: 0.56–0.93], respectively [2, 12, 16, 17, 21, 29, 31, 33,34,35]. In most of the studies, the AUC of the PRISM-3 was higher than PIM-3. A study found that the AUC of PRISM-3 was significantly higher than the PIM-3 (P = 0.04) [21], which is in line with our findings. Moreover, two studies reported poor discrimination measures for the PRISM-3 scoring system (AUC =0.667 [12] and 0.56 [33]), which might be due to specific conditions of their study sample (e.g., children receiving extracorporeal support for respiratory failure).

Table 5 Published evaluation studies of different versions of the two models in PICU
Full size table

In some studies, the Hosmer-Lemeshow test was used to evaluate the (lack of) concordance between observed versus predicted outcomes of the PIM-3 scoring system, which resulted in significant p-values (P = 0.003, P = < 0.001) [12, 41]. The PIM-3 performance was also evaluated in 49 PICUs in Argentina with 6602 patients aged between 1 month and 16 years and observed mortality rate was 8% (531/6602), whereas the predicted mortality by PIM-3 was 6.16% (407 deaths), moreover, the Hosmer-Lemeshow test showed disagreement between the predicted and observed mortality rates (χ2 = 135.63; P < 0.001) [36], supporting our result and Sankar and Wolfler studies [14, 41].

In our study the PRISM-3 model was well-calibrated, which is in line with findings provided by similar studies [12, 18, 19, 23, 27, 29, 32, 34, 35, 42]. However, there were also contradicting results showing poor calibration of these scoring systems. Aside from differences in the populations and selected sub-populations, this can also be due to the characteristics of the Hosmer-Lemeshow test as it is sensitive to the sample size (with larger sample size it tends to reject the null-hypothesis of agreement between the predicted and expected probabilities of the event) and cutoff points.

In several studies, it has been reported that a higher risk of mortality is associated with mechanical ventilation [2, 10, 23, 34]. The multivariable analysis of the Balkin et al. study showed that the ventilation support had the highest odds ratio among all covariates (OR: 2.1, 95% CI: 1.7–2.6), which is in line with our findings (P < 0.001) (11). This result was also confirmed by other studies, indicating the higher mortality rate for the patients admitted to the ICU with a higher number of organ failures [32, 35]. Also the prospective study in a pediatric oncology intensive care unit demonstrated that there is a significant relationship between mortality rate and diagnosis, the number of organ failures and ventilation support (P = 0.03, P < 0.001, P < 0.001, respectively) [23]. The presence of high urea and high creatinine, which often reflect low cardiac output or shock, suggests that renal function is an important prognostic indicator of mortality [2, 32].

Strengths and limitations

We conducted the analysis in a large heterogeneous multicenter cohort. In addition, we used a comprehensive battery of performance measures and conducted a rigorous internal validation using bootstrapping [43,44,45]. There are also some limitations in the present study which are important to mention: First, the original scoring systems were based on the worst value for each variable in the first 24 h, whereas in the current investigation, measures were obtained during the first hour of admission. However, by fitting the logistic regression model based on the scores ameliorates this limitation. Second, due to the retrospective study in some cases we did not have all the key variable required to calculate the scores. However, we used imputation to cope with the missing values. Third, although we considered all types of disease in our study, many patients with heart disease are directed to heart hospitals and are not in our cohort, which hence contains a limited proportion of heart patients. In this sense the cohort is not representative of those subgroups of critically ill patients. Future studies are needed for developing these models in other populations and for externally validating these models.

Conclusions

The prediction model based on PRISM-3 had superior predictive performance of that based on PIM-3 in discrimination, calibration, and accuracy of predicted probabilities. Further large validation studies are needed to consolidate these findings.

Availability of data and materials

The datasets used and/or analyzed during the current study available from the corresponding author on reasonable request.

References

  1. Goshayeshi L, Hoseini B, Yousefli Z, Khooie A, Etminani K, Esmaeilzadeh A, et al. Predictive model for survival in patients with gastric cancer. Electron Physician. 2017;9(12):6035–42. https://doi.org/10.19082/6035. Epub 2018/03/22.

  2. Balkin EM, et al. Intensive care mortality prognostic model for pediatric pulmonary hypertension. Pediatr Crit Care Med. 2018;19(8):733–40.

    Article  Google Scholar 

  3. Aluvaala J, et al. A systematic review of neonatal treatment intensity scores and their potential application in low-resource setting hospitals for predicting mortality, morbidity and estimating resource use. Syst Rev. 2017;6(1):248.

    Article  Google Scholar 

  4. Dickson KE, et al. Every newborn: health-systems bottlenecks and st rategies to accelerate scale-up in countries. Lancet. 2014;384(9941):438–54.

    Article  Google Scholar 

  5. Pollack MM, Patel KM, Ruttimann UE. PRISM III: an updated pediatric risk of mortality score. Crit Care Med. 1996;24(5):743–52.

    Article  CAS  Google Scholar 

  6. Pollack MM, Ruttimann UE, Getson PR. Pediatric risk of mortality (PRISM) score. Crit Care Med. 1988;16(11):1110–6.

    Article  CAS  Google Scholar 

  7. Shann F, et al. Paediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive Care Med. 1997;23(2):201–7.

    Article  CAS  Google Scholar 

  8. Visser IH, et al. Mortality prediction models for pediatric intensive care: comparison of overall and subgroup specific performance. Intensive Care Med. 2013;39(5):942–50.

    Article  Google Scholar 

  9. Thukral A, Lodha R, Irshad M, Arora NK. Performance of Pediatric Risk of Mortality (PRISM), Pediatric Index of Mortality (PIM), and PIM2 in a pediatric intensive care unit in a developing country. Pediatr Crit Care Med. 2006;7(4):356–61.

    Article  Google Scholar 

  10. Straney L, et al. Paediatric index of mortality 3: an updated model for predicting mortality in pediatric intensive care. Pediatr Crit Care Med. 2013;14(7):673–81.

    Article  Google Scholar 

  11. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45. Epub 1988/09/01.

  12. Tyagi P, Tullu MS, Agrawal M. Comparison of pediatric risk of mortality III, pediatric index of mortality 2, and pediatric index of mortality 3 in predicting mortality in a pediatric intensive care unit. J Pediatr Intensive Care. 2018;7(04):201–6.

    Article  Google Scholar 

  13. Burns JP, et al. Epidemiology of death in the PICU at five U.S. teaching hospitals*. Crit Care Med. 2014;42(9):2101–8.

    Article  Google Scholar 

  14. Wolfler A, et al. The importance of mortality risk assessment: validation of the pediatric index of mortality 3 score. Pediatr Crit Care Med. 2016;17(3):251–6.

    Article  Google Scholar 

  15. Slater A, Shann F. The suitability of the pediatric index of mortality (PIM), PIM2, the pediatric risk of mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in Australia and New Zealand. Pediatr Crit Care Med. 2004;5(5):447–53.

    Article  Google Scholar 

  16. Schlapbach LJ, et al. Prediction of pediatric sepsis mortality within 1 h of intensive care admission. Intensive Care Med. 2017;43(8):1085–96.

    Article  Google Scholar 

  17. Gemke RJ, van Vught JA. Scoring systems in pediatric intensive care: PRISM III versus PIM. Intensive Care Med. 2002;28(2):204–7.

    Article  Google Scholar 

  18. Gonçalves J-P, et al. Performance of PRISM III and PELOD-2 scores in a pediatric intensive care unit. Eur J Pediatr. 2015;174(10):1305–10.

    Article  Google Scholar 

  19. Volakli Ε, Mantzafleri P, Sdougka M. Pediatric risk of mortality (PRISM III-24) performance in a Greek pediatric intensive care unit. Greek E-J Perioper Med. 2013;11:31–43.

    Google Scholar 

  20. Brady AR, et al. Assessment and optimization of mortality prediction tools for admissions to pediatric intensive care in the United Kingdom. Pediatrics. 2006;117(4):e733–42.

    Article  Google Scholar 

  21. Jacobs A, et al. Performance of pediatric mortality prediction scores for PICU mortality and 90-day mortality. Pediatr Crit Care Med. 2019;20(2):113–9.

    Article  Google Scholar 

  22. Niederwanger C, et al. Comparison of pediatric scoring systems for mortality in septic patients and the impact of missing information on their predictive power: a retrospective analysis. PeerJ. 2020;8:e9993.

    Article  Google Scholar 

  23. Sayed HA, Ali AM, Elzembely MM. Can pediatric risk of mortality score (PRISM III) be used effectively in initial evaluation and follow-up of critically ill cancer patients admitted to pediatric oncology intensive care unit (POICU)? A prospective study, in a tertiary cancer center in Egypt. J Pediatr Hematol Oncol. 2018;40(5):382–6.

    Article  Google Scholar 

  24. Rady H, et al. Prediction of stress related gastrointestinal bleeding in critically III children using prism III score. J Anesth Crit Care Open Access. 2014;1(4):00023.

    Google Scholar 

  25. Nasser MM, et al. Reliability of pediatric risk of mortality III (PRISM III) and pediatric index of mortality 3 (PIM-3) scores in the pediatric intensive care unit of el-hussein university hospital. Al-Azhar J Pediatr. 2020;23(3):1084–71.

    Google Scholar 

  26. Varma A, et al. Prediction of mortality by pediatric risk of mortality (PRISM III) score in tertiary care rural hospital in India. Int J Contemp Pediatr. 2017;4:322–31.

    Article  Google Scholar 

  27. Siddique AW, et al. Mortality risk assessment in pediatric intensive care unit of a developing country using prism score. Pakistan Armed Forces Med J. 2019;69(3):690–5.

    Google Scholar 

  28. Ruangnapa K, et al. Validation of a modified pediatric risk of mortality III model in a pediatric intensive care unit in Thailand. Pediatr Respirol Crit Care Med. 2018;2(4):65.

    Article  Google Scholar 

  29. Jung JH, Sol IS, Kim MJ, Kim YH, Kim KW, Sohn MH. Validation of Pediatric Index of Mortality 3 for Predicting Mortality among Patients Admitted to a Pediatric Intensive Care Unit. Acute Crit Care. 2018;33(3):170–7. https://doi.org/10.4266/acc.2018.00150. Epub 2018/08/01.

  30. Lee OJ, et al. Validation of the pediatric index of mortality 3 in a single pediatric intensive care unit in Korea. J Korean Med Sci. 2017;32(2):365–70.

    Article  Google Scholar 

  31. Qiu J, Lu X, Wang K, Zhu Y, Zuo C, Xiao Z. Comparison of the pediatric risk of mortality, pediatric index of mortality, and pediatric index of mortality 2 models in a pediatric intensive care unit in China: A validation study. Medicine (Baltimore). 2017;96(14):e6431. https://doi.org/10.1097/md.0000000000006431. Epub 2017/04/07.

  32. Bai Z, et al. Effectiveness of predicting in-hospital mortality in critically ill children by assessing blood lactate levels at admission. BMC Pediatr. 2014;14(1):83.

    Article  Google Scholar 

  33. Barbaro RP, et al. Evaluating mortality risk adjustment among children receiving extracorporeal support for respiratory failure. ASAIO J. 2019;65(3):277–84.

    Article  Google Scholar 

  34. de Araujo Costa G, et al. Application of the Pediatric Risk of Mortality Score (PRISM) score and determination of mortality risk factors in a tertiary pediatric intensive care unit. Clinics. 2010;65(11):1087.

    Article  Google Scholar 

  35. El Hamshary AAE, et al. Prevalência da falência de múltiplos órgãos na unidade de terapia intensiva pediátrica: comparação dos escores Pediatric Risk of Mortality III e Pediatric Logistic Organ Dysfunction para predição de mortalidade. Rev Bras Ter Intensiva. 2017;29(2):206–12.

    Article  Google Scholar 

  36. López MdPA, et al. Performance of the pediatric index of mortality 3 score in PICUs in Argentina: a prospective, national multicenter study. Pediatr Crit Care Med. 2018;19(12):e653.

    Article  Google Scholar 

  37. Khajeh A, Noori NM, Reisi M, Fayyazi A, Mohammadi M, Miri-Aliabad G. Mortality risk prediction by application of pediatric risk of mortality scoring system in pediatric intensive care unit. Iran J Pediatr. 2013;23(5):546–50.

    PubMed  PubMed Central  Google Scholar 

  38. Bilan N, Galehgolab BA, Emadaddin A, Shiva S. Risk of mortality in pediatric intensive care unit, assessed by PRISM-III. Pak J Biol Sci. 2009;12(6):480–5. https://doi.org/10.3923/pjbs.2009.480.485. Epub 2009/07/08.

  39. Ramazani J, Hosseini M. Comparison of the predictive ability of the pediatric risk of mortality III, pediatric index of mortality3, and pediatric logistic organ dysfunction-2 in medical and surgical intensive care units. J Compr Pediatr. 2019;10(2).

  40. Saidi H, et al. Validation of pediatric index of mortality 3 in a single referral pediatric intensive care unit in Iran. Arch Pediatr Infect Dis. 2021; In Press.

  41. Sankar J, et al. Comparison of outcomes using pediatric index of mortality (PIM)-3 and PIM-2 models in a pediatric intensive care unit. Indian Pediatr. 2018;55(11):972–4.

    Article  Google Scholar 

  42. Ramazani J, Hosseini M. Comparison of the predictive ability of the pediatric risk of mortality III, pediatric index of mortality, and pediatric logistic organ dysfunction in medical and surgical intensive care units. J Compr Pediatr. (In Press).

  43. Rahmatinejad Z, Tohidinezhad F, Rahmatinejad F, Eslami S, Pourmand A, Abu-Hanna A, et al. Internal validation and comparison of the prognostic performance of models based on six emergency scoring systems to predict in-hospital mortality in the emergency department. BMC Emerg Med. 2021;21(1):68. https://doi.org/10.1186/s12873-021-00459-7. Epub 2021/06/12.

  44. Hoseini B, Rahmatinejad Z, Goshayeshi L, Bergquist R, Golabpour A, Ghaffarzadegan K, et al. Colorectal Cancer in North-Eastern Iran: a retrospective, comparative study of early-onset and late-onset cases based on data from the Iranian hereditary colorectal cancer registry. BMC Cancer. 2022;22(1):48. https://doi.org/10.1186/s12885-021-09132-5. Epub 2022/01/10.

  45. Rahmatinejad Z, Tohidinezhad F, Reihani H, Rahmatinejad F, Pourmand A, Abu-Hanna A, et al. Prognostic utilization of models based on the APACHE II, APACHE IV, and SAPS II scores for predicting in-hospital mortality in emergency department. Am J Emerg Med. 2020;38(9):1841–6. https://doi.org/10.1016/j.ajem.2020.05.053. Epub 2020/08/03.

Download references

Acknowledgements

This study was part of the first author MSc thesis and the authors would like to acknowledge Mashhad University of Medical Sciences for financial support.

Ethical issues

The permission was obtained from the Ethics Committee of Mashhad University of Medical Sciences.

Funding

This study was funded by Mashhad University of Medical Sciences, Mashhad, Iran (grant ID: 990511).

Author information

Author notes

  1. Zahra Rahmatinejad and Fatemeh Rahmatinejad contributed equally as joint lead authors in this work.

Authors and Affiliations

  1. Department of Medical Informatics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran

    Zahra Rahmatinejad, Fariba Tohidinezhad & Saeid Eslami

  2. Department of Medical Records and Health Information Technology, School of Paramedical Sciences, Mashhad University of Medical Sciences, Mashhad, Iran

    Fatemeh Rahmatinejad

  3. Pediatric Intensive Care, Department of Pediatrics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran

    Majid Sezavar

  4. Department of Medical Informatics, Amsterdam UMC - Location AMC, University of Amsterdam, Amsterdam, the Netherlands

    Ameen Abu-Hanna & Saeid Eslami

Authors
  1. Zahra Rahmatinejad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fatemeh Rahmatinejad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Majid Sezavar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fariba Tohidinezhad
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ameen Abu-Hanna
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Saeid Eslami
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZR, FR, AAH, and SE contributed to the study design. All authors (ZR, FR, MS, FT, AAH, and SE) contributed to data gathering and interpretation of the results. ZR, and FR performed analyses and wrote the first draft of the manuscript. AAH edited the final version of the manuscript. All authors (ZR, FR, MS, FT, AAH, and SE) read, commented, and approved the final manuscript.

Corresponding author

Correspondence to Saeid Eslami.

Ethics declarations

Ethics approval and consent to participate

The study protocol was approved by the institutional review board of the Mashhad University of Medical Sciences (Number: R.MUMS.REC.1398.011) which is conformed to the Declaration of Helsinki principles. The need for informed consent was waived by the Ethics Committee of Mashhad University of Medical Sciences because of the nature of the study and the analysis used anonymous clinical data.

Consent for publication

Not applicable.

Competing interests

There is no conflict of interest to declare.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rahmatinejad, Z., Rahmatinejad, F., Sezavar, M. et al. Internal validation and evaluation of the predictive performance of models based on the PRISM-3 (Pediatric Risk of Mortality) and PIM-3 (Pediatric Index of Mortality) scoring systems for predicting mortality in Pediatric Intensive Care Units (PICUs). BMC Pediatr 22, 199 (2022). https://doi.org/10.1186/s12887-022-03228-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12887-022-03228-y

Keywords

BMC Pediatrics

ISSN: 1471-2431

Contact us