Table 4 GRADE quality evaluation

Fall et al. 1998

Observational study–Low quality (2)

Large Magnitude of effect: Not present in this study. No reason to increase the grade (0)

Dose–response gradient: Not present in this study. No reason to increase the grade (0)

All plausible confounders or other biases increase our confidence in the estimated effect: There is a possibility that parents received education after their child had been hospitalized, which would have reduced the effect seen (+1)

Risk of Bias: Very serious, concern about selection process used and vague inclusion criteria (−2)

Inconsistency: The confidence interval showed no true overlap between the pre and post-test scores. No reason to down grade since the internal consistency did not appear to be in doubt (0)

Indirectness: Indirect measures, looking at knowledge of parents. Reasonable to downgrade (−1)

Imprecision: Confidence Interval is not narrow and there is a small sample size. Reasonable to downgrade (−1)

Publication bias: Small observation study, Likely to have publication bias (−1)

Very low (1)

Sullivan-Bolyai, et al. 2012

Randomized control trial: High quality (4)

Large Magnitude of effect: Not present (0)

Dose–response gradient: Not present (0)

All plausible confounders or other biases increase our confidence in the estimated effect: Not present (0).

Risk of Bias: Serious risk, lack of blinding and proper allocation concealment (−1)

Inconsistency: Serious inconsistency, low p-values with small F-values, indicating possible intragroup variability (−1)

Indirectness: Serious indirectness, study looked at self-efficacy and knowledge, no mention of patient-centered outcomes (−1)

Imprecision: Unable to appropriately determine confidence intervals given the limited information provided in the paper. Down-grade given the small sample size and the minimal improvement in scores when experimental compared to control arm (−1)

Publication Bias: Small pilot study, like to have publication bias since other studies with negative findings are not likely to be published. Reason to downgrade (−1)

Very low (1)

Thompson et al. 2007

Observational study–Low quality study (2)

Large Magnitude of effect: Not really present in this study. No reason to increase grade (0)

Dose–response gradient: Not really present in this study. No reason to increase grade (0)

All plausible confounders or other biases increase our confidence in the estimated effect: Not really present in this study. No reason to increase grade (0)

Risk of Bias: Serious risk of bias; concern about the fact that a majority of the data came from one location, additionally concern about the selection within the population regarding those who visit the locations where the kiosks were located (−1)

Inconsistency: Confidence intervals that were presented were narrow, and showed an effect that was consistent. No reason to downgrade (0)

Indirectness: Very indirect measures; looking at possibility of using the information instead of actually seeing if the information presented in the kiosks would be used (−2)

Imprecision: Confidence intervals were narrow and consistent. Sample size is large, so it is reasonable to capture patterns. No reason to downgrade (0)

Publication Bias: Study is rather large, and the findings are a reasonable mixture of positive and negative findings. No reason to down-grade (0)

Very Low (1)

Wenninger et al. 2000

Randomized control trial–High study quality (4)

Large Magnitude of effect: There was not a large magnitude of effect noted (0)

Dose–response gradient: Not really present in this study. No reason to increase grade (0)

All plausible confounders or other biases increase our confidence in the estimated effect: Not likely in this study. No reason to increase grade.(0)

Risk of Bias: No serious limitations, low risk of bias from some of the key areas. No reason to downgrade (0)

Inconsistency: Results were consistent, and the statistical F values showed a reasonable effect. No reason to downgrade (0)

Indirectness: Study employed a scale to measure disease severity, although this was not translated to direct clinical outcomes. The study also looked at quality of life and coping skills, rather indirect measures (−1)

Imprecision: The results, although positive, showed some variability through the t-scores for the disease severity scale, which may include the change in scores seen in the control group. Consider downgrading for imprecision (−1)

Publication Bias: No reasonable for publication bias, results showed some effect, although some of them were not statistically significant there is some clinical utility to them (0)

Low quality study (2)

Huss et al. 2003

Randomized control trial–High quality (4)

Large Magnitude of effect: Not present in the study. No reason to upgrade (0)

Dose–response gradient: Not present in the study. No reason to upgrade (0)

All plausible confounders or other biases increase our confidence in the estimated effect: This is possible given that since the confounding of asthma severity may cause residual biases against an effect. Reasonable to slightly rate up (+1)

Risk of Bias: Serious risk of bias. There was a lack of appropriate accounting of patients. There also is some selective reporting of outcomes (no real information on absolute symptom reduction). Reasonable to downgrade (−1)

Inconsistency: The estimate of effect was consistent with findings in other studies. Additionally there is some concern about the lack of appropriate controlling for possible confounders. Reasonable to downgrade (−1)

Indirectness: Study did try to measure disease severity through symptoms, although this is mixed in with some knowledge measures that were rather indirect. Would slightly downgrade for the indirect measures (−1)

Imprecision: The confidence intervals are wide with some overlap between the effect seen in the intervention and the control groups. Reasonable to downgrade (−1).

Publication Bias: The mixed nature of the results. The study was published in a reasonable journal. No reason to downgrade (0)

Very low quality study (1)

Guendelman et al. 2002

Randomized control trial- High quality (4)

Large Magnitude of effect: Not truly present in this study. No reason to upgrade (0)

Dose–response gradient: Not truly present in this study. No reason to upgrade (0)

All plausible confounders or other biases increase our confidence in the estimated effect: The possibile confounders of asthma severity would have worked in the direction of the effect, so there is no reason to upgrade the rating (0)

Risk of Bias: Low risk of bias from a few of the key criteria. There was good allocation concealment in place, although there was some loss to follow-up of some participants. No reason to downgrade (0)

Inconsistency: No reason to downgrade. Results are consistent throughout the study, and they are similar to other studies (0)

Indirectness: The study looked at disease severity, actual symptoms, ED visits, and missed days of school. These are very direct measures of the clinical effect of disease. No reason to downgrade (0)

Imprecision: The confidence intervals for several of the odds ratios are wide. It is reasonable to downgrade for the repeatedly wide confidence intervals (−1)

Publication Bias: No reason for possible publication bias given the thorough nature of the study (0)

Moderate quality study (3)

Shegog et al. 2001

Randomized control trial- High quality study (4)

Large Magnitude of effect: Not present in the study, so no reason to increase the grading (0)

Dose–response gradient: No present in this study. No reason to increase the grading (0)

All plausible confounders or other biases increase our confidence in the estimated effect: The difference between the intervention and control group, on the basis of asthma severity was not statistically significant, although this clear difference in terms of numbers would have made the intervention arm more likely to have issues with asthma, and likely more education. This confounding factor would have worked with the intervention, so there is no reason to increase the grade (0)

Risk of Bias: Serious risk of bias. There is concern about the use of allocation concealment in the study, as well as the randomization procedure used for the study (−1)

Inconsistency: The results seem to be consistent throughout the study. No reason to downgrade (0)

Indirectness: Indirect measures of knowledge were used, without any correlation to disease outcomes. Would downgrade (−1)

Imprecision: The confidence intervals were narrow, although there is some overlap between the intervention and control groups’ intervals in the knowledge based assessments. This overlap raises some question about the imprecision (−1)

Publication Bias: No reason to consider publication bias. The results were a mixture of positive and non-significant results (0)

Very low Quality study (1)

Horan et al. 1990

Randomized control trial with matching- High quality study

Large Magnitude of effect: Not present in this study (0)

Dose–response gradient: Not present in this study (0)

All plausible confounders or other biases increase our confidence in the estimated effect: The slight difference in disease knowledge at base line (more knowledge in the intervention group) would have supported the effect seen instead of working against the seen effect. No reason to increase grade (0)

Risk of Bias: Serious risk of bias. There is concern about the selection bias introduced through the matching process. It is reasonable to downgrade (−1)

Inconsistency: The reasonably large F values show solid internal consistency for the study. No reason to downgrade (0)

Indirectness: There was a direct clinically pertinent measure in this study, blood glucose levels. However, there was some indirect measures as well, knowledge and problem-solving. There is reason to downgrade (−1)

Imprecision: The F values show some reasonable effect with slower intragroup variability. No reason to downgrade (0)

Publication Bias: No reason to consider publication bias. The results had some reasonable support of their internal consistency through their F-values. The effect seen was small but reasonable.

Low quality study (2)

Dragone et al. 2002

Randomized control trial- High quality study (4)

Large Magnitude of effect: Not present in this study (0)

Dose–response gradient: Not really present in this study (0)

All plausible confounders or other biases increase our confidence in the estimated effect: The confounders that may be present would work with the effect seen in the study, so there is little likelihood that this effect would be an underestimation (0)

Risk of Bias: No serious risk for bias. There are slight issues with the measurement tools. No serious reason to downgrade (0)

Inconsistency: Internal consistency seems to rather solid give the high F-values. Results are consistent with other studies, and they are supported by internal consistency. No reason to downgrade (0)

Indirectness: Very indirect measures were used, looked at mental understanding of disease and knowledge (−2)

Imprecision: The high F-values support the strength of the analysis that was performed. There is minimal concern for the precision of analysis (0)

Publication Bias: The results were mixed in nature, not showing much change in the events from the interviews and modest effect for knowledge. No reason to consider publication bias (0)

Low quality study (2)

Large Magnitude of effect:

Dose–response gradient:

All plausible confounders or other biases increase our confidence in the estimated effect:

Risk of Bias:

Inconsistency:

Indirectness

Imprecision:

Publication Bias:

Krishna et al. 2003

Randomized control trial- High quality study (4)

Large Magnitude of effect: There was a small effect seen in terms of knowledge scores that improved (0)

Dose–response gradient: There was no dose–response curve as all the intervention groups received the same degree of intervention. No reason to increase the rating (0)

All plausible confounders or other biases increase our confidence in the estimated effect: One potential confounder that may be present, caregiver education, would work to minimalize the effect that is seen, so we could be slightly more confident in the effect that is seen (+1)

Risk of Bias: No serious concern for risk. There is some minimal concern about the measurements used, but this does not provide enough reason to downgrade (0)

Inconsistency: Although the confidence intervals are narrow, there is overlap in several of the confidence intervals between the control and intervention groups. This raises some concern about the internal consistency of the study (−1)

Indirectness: There was an appropriate mixture of indirect measures, knowledge scores, with clinically relevant outcomes, steroid dosage and emergency department visits. No reason to downgrade given the use of several clinically important outcomes (0)

Imprecision: The confidence intervals that were presented appear to be sufficiently narrow. No reason to downgrade (0)

Publication Bias: The results were mostly positive, yet the study was thorough and included clinically relevant outcomes, so the risk of publication bias seems minimal. The impact of the journal that published the study would argue that the study was rigorous evaluated. No reason to downgrade.

High quality study (4)

Homer et al. 2002

Randomized control trial- High quality study (4)

Large Magnitude of effect: There was not that large of an effect seen in the study. The betas that were determined did not show that strong of a relationship. No reason to increase the grading (0)

Dose–response gradient: There was a slight dose–response gradient seen in some of the regression models that were performed (+1)

All plausible confounders or other biases increase our confidence in the estimated effect: The possible confounders in the study were included in the analysis of variance. Since they were incorporated into the analysis, it seems difficult to see them working against or for the effect seen. No reason to increase the grade (0)

Risk of Bias: Serious risk of bias. There is some serious concern about the different sites used for recruitment, different clinical settings with possibly different patient populations served. Reasonable to downgrade (−1)

Inconsistency: The large F-value shows some small intragroup variability to support the internal validity of the study. No reason to downgrade (0)

Indirectness: There is a mixture of indirect measures, knowledge, and several clinical outcomes, emergency department visits and asthma severity scores. There is slightly more indirect measures, so there is some reason to downgrade slightly (−1)

Imprecision: There is concern about the precision in the study. There is no clear way to look at regression analyses that were performed. The analysis of variance did show some reasonable intragroup study precision. Reasonable to downgrade for imprecision (−1).

Publication Bias: There is little reason to consider publication bias. The results included some non-significant and significant findings that appeared to be appropriate. No reason to downgrade (0)

Low quality study (2)

Swallow et al. 2014

Randomized control trial- High quality study (4)

Large Magnitude of effect: There was not a large effect seen in this study (0)

Dose–response gradient: There was not a clear dose–response gradient observed in this study (0)

All plausible confounders or other biases increase our confidence in the estimated effect: There was the confounder of socioeconomic status, which showed the intervention group having a lower socioeconomic status. This would have worked against the effect, so our confidence that in the effect seen would be increased (+1)

Risk of Bias: Serious concern about selection bias introduced through the lack of blinding in the study. Also concerned about the handling of missing data in the analysis. Reasonable to downgrade (−1)

Inconsistency: There is great variability in the intraclass coefficients presented in the study. Reasonable to downgrade for some concern about the internal consistency of the study (−1).

Indirectness: Serious concern about the indirect measures that were used in the study, looking at the results of surveys and scales to assess parental management ability and father support (−1)

Imprecision: The confidence intervals that are presented appear to vary widely, with overlap between the control and intervention groups. Reasonable to downgrade for precision in the study (−1)

Publication Bias: The study was thorough, including some positive and non-significant findings. There seems to be little reason to downgrade for publication bias (0)

Very low quality study (1)