Overall guideline quality
Previous studies assessing quality assessment of pediatric CPGs are outdated or only focused on a specific field [21,22,23,24]. Isaac et al. conducted a study in 2011 to evaluate the quality of development and reporting of 28 CPGs developed or endorsed by AAP. After assessment with AGREE II, they showed that the CPGs achieved an overall mean score of 55%, which is lower than the present study. Furthermore, they reported 29% of the CPGs with an overall score of < 50%, while this proportion decreased in the present study . These results suggest that the overall quality of pediatric CPGs improved since 2011. However, the number of CPGs reaching high quality (receiving the “recommend” level) did not change significantly, compared with before . Xie et al. appraised pediatric CPGs related to community-acquired pneumonia published from January 2000 to March 2015. In their study, 30% of CPGs achieved the “recommended” levels, 40% of CPGs were “recommended with modifications”, and 30% of CPGs were “not recommended” . Generally, based on existing research, the overall quality of pediatric CPGs improved compared to early CPGs [21, 32]. However, there were still few CPGs that reached a high-quality level. Moreover, the overall quality score was still inadequately compared to the quality evaluation for other recent CPGs focused on adults. Most of the studies that focused on adult CPGs reported a mean overall AGREE II scores of 4.77–5.97 in 7 points (68.21–85.35%), and 8.2–50.0% of them could reach the “recommend” level [33,34,35]. A study published in 2018 analyzed 89 CPGs on adult critical care, and reported a mean overall score of 83%, which was higher than this review . The study by Madera et al. suggested that 50% of the eight adult CPGs on screening and diagnosis of oral cancer were assessed as “recommend” and the other 50% were assigned as “recommended with modifications” . Compared with CPGs for adults, the quality of pediatric CPGs still needs to be improved.
Quality of domains
Compared with other studies using the AGREE II assessment, the present study also revealed that “applicability” and “rigor of development” domains had poorer quality [21, 22, 35, 36]. A study of previous assessment of pediatric CPGs showed that “applicability”, “editorial independence”, and “stakeholder involvement” domains achieved the lowest mean scores, at 19, 40, and 42%, respectively . We also compared the scores of each domain among CPGs with different recommendation levels to determine which domains affect the recommendation level. As shown in Fig. 2, the CPGs that achieved lower recommendation levels were insufficient in “applicability” and “rigor of development”, which indicated these domains affected the overall quality of pediatric CPGs.
The “applicability” domain mainly focuses on the barriers and facilitators to apply the CPG . This domain required CPGs to consider facilitators and barriers in the application, and provide advice or tools for different age groups and regions. The clinical manifestations, progress, and outcomes of pediatric diseases are different from those of adult diseases. Therefore, before applying a CPG, it is necessary to evaluate its quality and scope of application. The study of Boluyt et al. was a great example of adopting CPGs . They conducted a systematic review of CPGs and assessed the quality and applicability of the CPGs. Furthermore, they synthesized the expert opinions to determine the CPGs that can be used in local clinical practice .
The “rigor of development” domain is the key to the development of a qualified CPG. This domain relates to gathering and synthesizing the evidence, promoting recommendations and update schedules of CPGs . The AGREE II manual  and RIGHT checklist  provide various suggestions in CPG development and reporting, such as systematic methods, evidence criteria, review procedure, and update schedule, which should be consulted and followed in the proposal, development, report, review, and update procedures of a CPG.
Recently, several studies raised the concern that conflict of interest could affect the quality of CPGs [38,39,40,41]. However, only limited CPGs described the management of financial conflicts of interest . Komesaroff et al. proposed the concept of “conflicts of interests” as “the condition that arises when two coexisting interests directly conflict with each other: that is when they are likely to compel contrary and incompatible outcomes” ; while Grundy et al. and Wiersma et al. suggested “non-financial conflicts of interests” should also receive awareness in health and medicine [41, 42]. The AGREE II provides a domain as “editorial independence” to evaluate whether the funding bodies have influenced the content and whether conflicts of interests of CPG development group members have been recorded and addressed . Our study showed that the “editorial independence” domain achieved a mean score of only 35.26% for pediatric CPGs. In addition, several previous studies highlighted that “editorial independence” domain of AGREE II in pediatric CPGs had inappropriate quality (a mean score of 17–48%) [19,20,21]. Thus, the potential conflicts of interests in CPG development should be disclosed and reviewed carefully. Independent committees should also be engaged for evaluation and management [18, 40].
Influential factors of quality
Some studies showed a significant improvement in CPGs’ quality under organizations or groups’ responsibility [8, 20]. According to the study of Font-Gonzalez et al., CPGs under organizations or groups’ responsibility were more likely to have high quality . In the present study, only a few CPGs (14.3%) were not conducted by organizations or groups. Reliable organizations or groups can complete the CPG development procedures, use appropriate methods, and report in a more complete manner, which might be relatively difficult for an individual or small team . Furthermore, a small team might lack the skills or training in developing CPGs as compared with large organizations or groups .
Previous studies suggested that a non-evidence-based method in CPG development might significantly affect quality . In the present study, one-quarter of CPGs did not use evidence-based methods, and we found that non-evidence-based methods had significant influence in nearly all domains. The evidence-based method was important in CPG development and clinical decision-making . By using an evidence-based method, we could systematically search and summarize previous research, reducing the limitations and bias .
Several studies suggested CPGs developed in regions with different economic development statuses might influent the quality of CPGs [22, 43]. The present study also found that CPGs developed by developing countries or regions had poorer quality in domains related to “scope and purpose”, “stakeholder involvement”, “applicability”, and “editorial independence”. Also, we found that most of the CPGs with poor quality developed by developing countries or regions did not follow a strict and comprehensive development procedure; and some of them did not use the evidenced-based method, which might influence quality. Most of the CPGs with high quality were developed by countries or organizations with significant funding and resource. A previous study suggested that AAP’s internal CPGs had significantly higher total scores than endorsed CPGs . These CPGs with high quality were developed under a strictly completed, evidence-based CPG development procedure. Additionally, the CPG committee consisted of clinical experts, methodologists, and others involved from different fields, improving the rigor in development and applicability in practice . For resource-limited developing countries, it might be a challenge to form a complete expert group to complete the CPG development procedure. One possible way of these regions was adapting existing high-quality CPGs . In addition, international collaboration could be an acceptable way of developing a CPG . However, as there were nuances in many healthcare systems worldwide that might preclude the direct deployment of international CPGs, agencies should consider CPG adaptations for their institutions. The process for guideline adaptation (ADAPTE) could create CPG versions, derived from existing CPGs, but modified to local settings, which is a cost-effective and less resource-intensive approach to CPG development . Recently, Dizon et al. suggested a standardized procedure to adopt, adapt or contextualize recommendations from existing CPGs of good quality, promoting the use of scarce resources more focused on implementation . These studies provided meaningful attempts at tailoring CPGs to the local context.
The present study had several limitations. Firstly, because the present study’s primary purpose was to evaluate the quality of recent pediatric CPGs, we only assessed CPGs published in the past 3 years, which limits the evaluation of the change in CPGs’ quality over time. Also, only English CPGs were included in this study; therefore, further research should analyze CPGs that were written in different languages when possible. Secondly, the AGREE II assessment was related to the personal judgment of reviewers, which might introduce selection bias. Thus, we conducted strict training and test assessment procedures. A re-assessment procedure was also performed to reduce selection bias. Finally, AGREE II has its inherent limitations. AGREE-II scores are dependent upon reporting, while some CPG committees may comply with the requirements but do not ultimately report. In addition, AGREE II only focuses on the quality in developing and reporting procedures of CPGs, but the evidence behind the recommendations cannot be evaluated. Thus, AGREE II is not sufficient to ensure that CPG recommendations are appropriate and accurate [13,14,15]. Several studies suggested that a new version of AGREE with an evaluation of CPGs’ contents should be proposed, which would require a great effort and collaboration [13,14,15]. We suggest health providers should closely follow new versions of well-developed tools for the appraisal of CPGs. Before that, health care providers should assess CPG quality using tools like AGREE II and evaluate CPG content and local adaptations before implying recommendations from a CPG [26, 50, 51]. Furthermore, different CPGs might contradict some recommendations, which cannot be solved by AGREE II alone. When these contradictions occur, health providers should review its contents and evidence. Thus, the decision to implement recommendations from CPGs requires careful considerations, including its quality, contents, adaptions, patients’ wishes, resources, feasibility, and fairness.