10-year trend in quantity and quality of pediatric randomized controlled trials published in mainland China: 2002–2011
BMC Pediatrics volume 13, Article number: 113 (2013)
Quality assessment of pediatric randomized controlled trials (RCTs) in China is limited. The aim of this study was to evaluate the quantitative trends and quality indicators of RCTs published in mainland China over a recent 10-year period.
We individually searched all 17 available pediatric journals published in China from January 1, 2002 to December 30, 2011 to identify RCTs of drug treatment in participants under the age of 18 years. The quality was evaluated according to the Cochrane quality assessment protocol.
Of 1287 journal issues containing 44398 articles, a total of 2.4% (1077/44398) articles were included in the analysis. The proportion of RCTs increased from 0.28% in 2002 to 0.32% in 2011. Individual sample sizes ranged from 10 to 905 participants (median 81 participants); 2.3% of the RCTs were multiple center trials; 63.9% evaluated Western medicine, 32.5% evaluated traditional Chinese medicine; 15% used an adequate method of random sequence generation; and 10.4% used a quasi-random method for randomization. Only 1% of the RCTs reported adequate allocation concealment and 0.6% reported the method of blinding. The follow-up period was from 7 days to 96 months, with a median of 7.5 months. There was incomplete outcome data reported in 8.3%, of which 4.5% (4/89) used intention-to-treat analysis. Only 0.4% of the included trials used adequate random sequence allocation, concealment and blinding. The articles published from 2007 to 2011 revealed an improvement in the randomization method compared with articles published from 2002 to 2006 (from 2.7% to 23.6%, p = 0.000).
In mainland China, the quantity of RCTs did not increase in the pediatric population, and the general quality was relatively poor. Quality improvements were suboptimal in the later 5 years.
Randomized controlled trials (RCTs) provide reliable evidence to guide clinical practice. However, difficulties in obtaining a guardian’s consent, obtaining research funding, and ethical concerns [1–3] often limit the conduction of RCTs in children. More attention should be paid to the quality of research in clinical trials evaluating drugs in children, which would lead to improvements in pediatric clinical practices. Al-Namankany et al.  showed that the proportion of studies in pediatric dentistry journals using random sequence generation was 28% and allocation concealment was 6%. Crocetti et al.  found that a large proportion of pediatric RCT reports in eight prominent journals between July 1, 2007 and June 30, 2008 had a high risk of bias. Hamm et al.  also found that more than half of a random sample of 300 pediatric RCTs published in 2007 had a high risk of bias. Wu et al.  found that only 6.6% of RCTs published in Chinese journals used adequate methods of random sequence generation. These previous studies provided insightful information on the quality of pediatric RCTs. However, they did not examine whether there was an improvement in quality nor give details of quantitative trends over a period.
In this study, we analyzed RCTs evaluating drug treatment in 17 Chinese pediatric journals in a recent 10-year period to elucidate information about the quantitative trends and quality of these studies in mainland China.
We individually searched currently available pediatric journals published from January 1, 2002 to December 30, 2011 in mainland China. There are a total of 17 pediatric medical journals, namely Chinese Pediatrics of Integrated Traditional and Western Medicine, Chinese Journal of Neonatology, Journal of China Pediatric Blood and Cancer, Journal of Clinical Pediatric Surgery, Chinese Journal of Evidence-Based Pediatrics, Journal of Clinical Pediatrics, Journal of Applied Clinical Pediatrics, Chinese Journal of Contemporary Pediatrics, Chinese Journal of Practical Pediatrics, Chinese Journal of Pediatrics, Chinese Journal of Pediatric Surgery, Chinese Journal of Obstetrics & Gynecology and Pediatrics, International Journal of Pediatrics, Chinese Pediatric Emergency Medicine, Journal of Pediatric Pharmacy, Chinese Journal of Perinatal Medicine, and Journal of Pediatrics of Traditional Chinese Medicine.
Inclusion and exclusion criteria
We included RCTs with participants under the age of 18 years and which used a pharmaceutical intervention. We excluded reviews, self-control studies and trials with more than two intervention groups.
Selection of studies and data extraction
One reviewer screened the titles and abstracts of every article that was published in the included journals. The full articles were obtained and further evaluated by the selection criteria outlined above. For the selected RCTs, two reviewers independently performed data extraction. The extracted data included: (1) the year of publication; (2) the general characteristics: the research institutions, multicenter or single center, sample size, comparability of baseline characteristics, and the funding resources, etc.; and (3) quality assessment (see Quality assessment section below). Discrepancies were resolved through discussion.
Evaluation of the quality of the research methods was based on the Cochrane quality assessment list . Two reviewers independently used the criteria for the evaluation process. The criteria were as follows: (1) random sequence generation; (2) allocation concealment; (3) blinding; (4) incomplete outcome data; (5) selective reporting; and (6) other sources of bias. Discrepancies were resolved through discussion. The details of assessment of bias are listed in Table 1.
Data management and analysis
Continuous variables were analyzed using mean values and standard deviations or median and interquartile range, while categorical variables were analyzed using percentages. The Student t-test and Mann–Whitney U test were used to test differences in continuous variables as appropriate, and the χ 2 test was used for proportions. Data management and analysis were performed with the SPSS v. 16.0 (SPSS Inc., Chicago, IL, USA).
After screening 44398 articles published in the 17 Chinese pediatric medical journals, a total of 1077 RCTs were included in the analysis (Figure 1). Over the 10-year period, the proportion of RCTs in mainland China increased from 0.28% in 2002 to 0.32% by 2011 (Figure 2). There was no significant increase in the number of RCTs conducted annually. Only 4.09% of the trials reported financial funding; however, we could not confirm that how many trials were sponsored by pharmaceutical companies because the information was not given. Individual study sample sizes ranged from 10 to 905 (median 81 participants). Only 0.6% (7/1077) of RCTs reported that calculation of sample size had been performed, and 2.3% (25/1077) of RCTs included multiple centers, with the number of participating centers ranging between two and ten. Nearly one-quarter (254/1077) of RCTs were conducted in teaching hospitals, 63.9% (688/1077) evaluated Western medicine, 32.5% (350/1077) evaluated traditional Chinese medicine, and 3.6% (39/1077) evaluated a combination of both traditional Chinese and Western medicine (Table 2). The top five categories of disease included respiratory (41.6%), neonatal (19.9%), digestive (14.7%), infection (5.7%), and neuromuscular (4.4%) diseases (Figure 3).
The results are presented in Table 3: 15% (161/1077) of RCTs used an adequate method of random sequence generation; 74.6% (803/1077) denoted “random allocation” in the study without a specific description; and 10.4% (113/1077) regarded quasi-randomization as correct randomization, and the most common methods for allocating participants used were the visiting sequence of inpatient/outpatient, the card number of the inpatient/outpatient number or patient birth date.
Only 1% (10/1077) of RCTs reported reasonable allocation concealment, including central allocation and sealed opaque envelopes; 99% (1067/1077) of trials did not indicate the allocation concealment method.
Only 0.6% (6/1077) of RCTs reported the blinding method (0.3% single-blind for participants, 0.2% double-blind for participants and investigators, 0.1% triple-blind for participants, investigators and assessors); 1.6% of RCTs (17/1077) reported blinding without full disclosure of the method; and 97.8% (1054/1077) did not mention the use of blinding in the trials.
Interventions and outcomes measurement
Only 1.1% (22/1077) of RCTs were placebo-controlled, 56.1% (604/1077) were open control and 42.8% (461/1077) were positive drug control. “Open control” was defined as having two groups with the same standard treatment with intervention in the treatment group (i.e., standard treatment + test drug versus standard treatment). “Positive drug control” was defined as the control group having an active control (i.e., test drug versus control drug, or standard treatment + test drug versus standard treatment + control drug). A positive result was reported in 95.5% (1028/1077) of trials; 4% (43/1077) of trials used laboratory data as indicators of efficacy and 58% (625/1077) used endpoint outcomes, while 38% (409/1077) used both laboratory data and endpoint outcomes. The follow-up period ranged from 7 days to 96 months, with a median of 7.5 months. Incomplete outcome data was reported in 8.3% (89/1077) of trials, of which 4.5% (4/89) used intention-to-treat analysis; 49.2% (530/1077) of trials reported adverse events, of which 52.6% (279/530) reported specific adverse events, and 47.4% (251/530) reported no adverse events; 1.7% (18/1077) of trials showed obvious selective reporting (outcomes described in the methodology were not reported in the results); in 98.3% (1059/1077) the possibility of selective reporting could not be evaluated because none of the included studies had been registered, and we were unable to compare the research with the protocol. In general, 0.4% (4/1077) reported all random sequence, allocation concealment, and blinding methods.
Bias from other sources
In 0.2% (2/1077) of RCTs, the reported baseline data in the treatment and control groups were not comparable.
We conducted a subgroup analysis to determine which factors influenced the quality of the included studies. The quality of RCTs conducted in multiple centers was superior to those of single centers regarding random sequence generation, allocation concealment, and blinding. The quality of allocation concealment and blinding but not random sequence generation, was also better in teaching hospitals compared with non-teaching hospitals. In contrast, incomplete outcome data was more likely in RCTs from multiple centers and in teaching hospitals. Trials with financial funding performed better for allocation concealment than trials without funding. However, we could not explore the influence of pharmaceutical industry funding on quality because sponsorship information was not reported. The RCTs published from the 2007 to 2011 revealed an improvement in the randomization method compared with 2002–2006 (from 2.7% to 23.6%). There was also a trend for improvement in allocation concealment (p = 0.053) (Table 4).
This study showed that the number and the quality of Chinese pediatric RCTs improved slightly during the past decade. It had been reported that the number of adult RCTs was 2.7 times that of pediatric RCTs in six medical journals . The number of pediatric RCTs increased from 0.4 to 16.9 per year, while adult RCTs increased from 4.71 to 90.5 per year [10–13]. Owing to a lack of pediatric RCTs, use of drugs in pediatric settings are often forced to extrapolate from the results of clinical trials in adults, which is inappropriate [10, 14]: on the one hand, there are many differences in physiological function, pharmacokinetics, and pharmacodynamics between children and adults, thus the administration of unevaluated pharmaceutics in children by extrapolation could have no effect or be harmful ; on the other hand, this would result in off-label drug use, which is relevant to the occurrence of adverse events . Reports indicate that the rate of off-label pharmaceutical use in pediatric settings ranges from 11% to 46% [17–23]. Recently, a survey suggested that the rate of off-label drug use in China is 78.96% in hospitalized children . In accordance with the prevalence of off-label drug use in pediatric settings, the risk of adverse effects has been reported to be three times as high as in adults .
There are five major reasons for the limited clinical trials of drugs in children: (1) the spectrum of diseases is narrow and the incidence of disease in children is low compared with adults ; (2) difficulties in obtaining consent from guardians limit the number of participants available for clinical trials [3, 14, 26]; (3) the unwillingness of researchers to conduct research as a result of the extensive pressures exerted by ensuring participants’ rights in trials and minimizing risk in clinical trials ; (4) the limited financial support from pharmaceutical companies because of the high cost of research and the low beneficial return; and (5) the lack of public policies supporting a re-evaluation of procedures for off-label drug use [11, 15].
There was a median of 81 participants in the published pediatric RCTs in China, which is lower than in international studies (median of 272 participants). In addition, the proportion reporting sample size calculation in Chinese pediatric RCTs was also lower than in international pediatric RCTs (4–65%) [11, 28] and adult RCTs (23–64%) [28–30]. The proportion of multiple center pediatric RCTs in China (2.3%) was also lower than the adult RCTs (28–67%) worldwide [11, 28].
Inadequate or unclear research methods of allocation concealment and random allocation signify low quality clinical trials and may exaggerate efficacy by as much as 30–41% [31–33]. Compared with the analysis of studies in pediatric dentistry conducted by Al-Namankany et al. , the proportion of studies with adequate randomization and blinding was low, while use of intention-to-treat analysis was similar to our study. Compared with adult RCTs, the proportion with adequate randomization in our study was similar to the international level (14–39%), and allocation concealment, blinding, and intention-to-treat analysis were at a lower level than the international standard (allocation concealment 13–40%, blinding 19–45%, intention-to-treat analysis 12–43%) [28–30, 34].
The main problems in the pediatric RCTs in China were as follows: (1) studies were often labeled “random” without giving details on random sequence generation, and some researchers erroneously regarded the quasi-random process as the correct method for randomization; (2) outcome measurements using blinding methodology were ignored, and explanations of the use of blinding methodologies were vague; (3) most RCTs used active drugs whose efficacy was unclear as control, and few used a placebo control; (4) RCTs focused on the evaluation of short-term efficacy, and had no discussion of long-term efficacy; (5) adverse effects were often ignored or not reported; and (6) there was lack of clinical trial registration making it difficult to monitor the quality of the conduction and reporting of trials. Further studies should overcome these drawbacks.
The quality of the published RCTs in China did show an increase in the reported random sequence generation methods in studies published from 2007 to 2011 compared with 2002 to 2006. The proportion with adequate use of randomized sequence generation methods increased from 2.7% to 23.6%. The following factors may have contributed to the increased use of randomized sequence generation: (1) a rapid development of evidence-based medicine in China; (2) greater awareness by clinicians of the importance of quality in clinical research for clinical decision-making; and (3) improved quality of research, with Consolidated Standards Of Reporting Trials(CONSORT) publications available to researchers to scrutinize standard RCTs. Compared with the quality of international clinical trials, only 25% of the pediatric trials published in China achieved adequate random sequence generation, which still leaves major opportunities for improvement. The quality of multiple center RCTs in China was better than in single center studies, which was consistent with international reports .
There were several limitations in our study: (1) assessment of the quality of RCTs was based on the description in the articles without verification from the original authors. However, a verification process suggested there would be no significant improvement in the assessment ; (2) we only included RCTs of drug interventions and excluded non-drug therapy (such as physical therapy or surgery); in addition, we also excluded more than two intervention groups, and as it is more difficult to control quality in these studies, our study may have overestimated the quality of pediatric RCTs in China; and (3) we only included trials published in pediatric professional journals, and other general medical journals could contain pediatric RCTs.
In mainland China, the quantity of RCTs did not increase over 10 years in the pediatric population, and the general quality remained relatively poor. There was an improvement in quality in the latest 5 years, but this was suboptimal.
Randomized controlled trials
Consolidated Standards Of Reporting Trials.
Budetti PP: Ensuring safe and effective medications for children. JAMA. 2003, 290: 950-951. 10.1001/jama.290.7.950.
Morales-Olivas FJ, Morales-Carpi C: Clinical trials in children. Rev Recent Clin Trials. 2006, 1: 251-258. 10.2174/157488706778250087.
Caldwell PH, Butow PN, Craig JC: Parents’ attitudes to children's participation in randomized controlled trials. J Pediatr. 2003, 142: 554-559. 10.1067/mpd.2003.192.
Al-Namankany AA, Ashley P, Moles DR, Parekh S: Assessment of the quality of reporting of randomized clinical trials in paediatric dentistry journals. Int J Paediatr Dent. 2009, 19: 318-324. 10.1111/j.1365-263X.2009.00974.x.
Crocetti MT, Amin DD, Scherer R: Assessment of risk of bias among pediatric randomized controlled trials. Pediatrics. 2010, 126: 298-305. 10.1542/peds.2009-3121.
Hamm MP, Hartling L, Milne A, Tjosvold L, Vandermeer B, Thomson D, Curtis S, Klassen TP: A descriptive analysis of a representative sample of pediatric randomized controlled trials published in 2007. BMC Pediatr. 2010, 22: 96-
Wu T, Li Y, Bian Z, Liu G, Moher D: Randomized trials published in some Chinese journals: how many are randomized?. Trials. 2009, 10: 46-10.1186/1745-6215-10-46.
Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. Edited by: Higgins J, Green S. 2011, The Cochrane Collaboration, Available from http://www.cochrane-handbook.org
Martinez-Castaldi C, Silverstein M, Bauchner H: Child versus adult research: the gap in high-quality study design. Pediatrics. 2008, 122: 52-57. 10.1542/peds.2007-2849.
Campbell H, Surry SA, Royle EM: A review of randomised controlled trials published in Archives of Disease in Childhood from 1982–96. Arch Dis Child. 1998, 79: 192-197. 10.1136/adc.79.2.192.
Cohen E, Uleryk E, Jasuja M, Parkin PC: An absence of pediatric randomized controlled trials in general medical journals, 1985–2004. J Clin Epidemiol. 2007, 60: 118-123. 10.1016/j.jclinepi.2006.03.015.
Cohen E, Goldman RD, Ragone A, Uleryk E, Atenafu EG, Siddiqui U, Mahmoud N, Parkin PC: Child vs adult randomized controlled trials in specialist journals: a citation analysis of trends, 1985–2005. Arch Pediatr Adolesc Med. 2010, 164: 283-288. 10.1001/archpediatrics.2009.291.
Wang G, Mao B, Xiong ZY, Fan T, Chen XD, Wang L, Liu GJ, Liu J, Guo J, Chang J, Wu TX, Li TQ: The quality of reporting of randomized controlled trials of traditional Chinese medicine: a survey of 13 randomly selected journals from mainland China. Clin Ther. 2007, 29: 1456-1467. 10.1016/j.clinthera.2007.07.023.
Menson EN, Walker AS, Gibb DM: Clinical trials in children. Lancet. 2004, 364: 2176-2177. 10.1016/S0140-6736(04)17581-8.
Roberts R, Rodriguez W, Murphy D, Crescenzi T: Pediatric drug labeling: improving the safety and efficacy of pediatric therapies. JAMA. 2003, 290: 905-911. 10.1001/jama.290.7.905.
Santos DB, Clavenna A, Bonati M, Coelho HL: Off-label and unlicensed drug utilization in hospitalized children in Fortaleza, Brazil. Eur J Clin Pharmacol. 2008, 64: 1111-1118. 10.1007/s00228-008-0543-1.
McIntyre J, Conroy S, Avery A, Corns H, Choonara I: Unlicensed and off label prescribing of drugs in general practice. Arch Dis Child. 2000, 83: 498-501. 10.1136/adc.83.6.498.
Conroy S, Choonara I, Impicciatore P, Mohn A, Arnell H, Rane A, Knoeppel C, Seyberth H, Pandolfini C, Raffaelli MP, Rocchi F, Bonati M, Jong G, de Hoog M, van den Anker J: Survey of unlicensed and off label drug use in paediatric wards in European countries. European Network for Drug Investigation in Children. BMJ. 2000, 320: 79-82. 10.1136/bmj.320.7227.79.
t Jong GW, Vulto AG, de Hoog M, Schimmel KJ, Tibboel D, van den Anker JN: Unapproved and off-label use of drugs in a children's hospital. N Engl J Med. 2000, 343: 1125-10.1056/NEJM200010123431515.
Dell'Aera M, Gasbarro AR, Padovano M, Laforgia N, Capodiferro D, Solarino B, Quaranta R, Dell'Erba AS: Unlicensed and off-label use of medicines at a neonatology clinic in Italy. Pharm World Sci. 2007, 29: 361-367. 10.1007/s11096-006-9081-z.
Palcevski G, Skocibusic N, Vlahovic-Palcevski V: Unlicensed and off-label drug use in hospitalized children in Croatia: a cross-sectional survey. Eur J Clin Pharmacol. 2012, 68: 1073-1077. 10.1007/s00228-012-1221-x.
Lindell-Osuagwu L, Korhonen MJ, Saano S, Helin-Tanninen M, Naaranlahti T, Kokki H: Off-label and unlicensed drug prescribing in three paediatric wards in Finland and review of the international literature. J Clin Pharm Ther. 2009, 34: 277-287. 10.1111/j.1365-2710.2008.01005.x.
Gavrilov V, Lifshitz M, Levy J, Gorodischer R: Unlicensed and off-label medication use in a general pediatrics ambulatory hospital unit in Israel. Isr Med Assoc J. 2000, 2: 595-597.
Zhang LL, Li YP, Hu D, Zeng LN, Huang L, Chen M, Yang CS: Off-label Drug Use in Pediatric Wards of West China Second University Hospital in 2010: A Cross Sectional Study. Chin J Evid-Based Med. 2012, 2: 161-167.
Crowley E, Williams R, Cousins D, Williams DC: Medication errors in children: a descriptive summary of medication error reports submitted to the United States Pharmacopeia. Curr Ther Res. 2001, 62: 627-640. 10.1016/S0011-393X(01)80069-2.
Smyth RL, Weindling AM, Weindling , 2: Research in children: ethical and scientific aspects. Lancet. 1999, 354: SII21-SII24.
Federman DD, Hanna KE, Rodriguez LL: Responsible Research: A Systems Approach to Protecting Research Participants. 2002, Washington, DC: National Academies Press
Hopewell S, Dutton S, Yu LM: The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ. 2010, 340: c723-10.1136/bmj.c723.
Sinha S, Ashby E, Jayaram R, Grocott MP: Quality of reporting in randomized trials published in high-quality surgical journals. J Am Coll Surg. 2009, 209: 565-571. 10.1016/j.jamcollsurg.2009.07.019. e1
Uetani K, Nakayama T, Ikai H, Yonemoto N, Moher D: Quality of reports on randomized controlled trials conducted in Japan: evaluation of adherence to the CONSORT statement. Intern Med. 2009, 48: 307-313. 10.2169/internalmedicine.48.1358.
Juni P, Altman DG, Egger M: Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ. 2001, 323: 42-46. 10.1136/bmj.323.7303.42.
Schulz KF, Chalmers I, Hayes RJ, Altman DG: Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995, 273: 408-412. 10.1001/jama.1995.03520290060030.
Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA: Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008, 336: 601-605. 10.1136/bmj.39465.451748.AD.
Kober T, Trelle S, Engert A: Reporting of randomized controlled trials in Hodgkin lymphoma in biomedical journals. J Natl Cancer Inst. 2006, 98: 620-625. 10.1093/jnci/djj160.
Bafeta A, Dechartres A, Trinquart L, Yavchitz A, Boutron I, Ravaud P: Impact of single centre status on estimates of intervention effects in trials with continuous outcomes: meta-epidemiological study. BMJ. 2012, 344: e813-10.1136/bmj.e813.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2431/13/113/prepub
The authors declare that they have no competing interests.
C-SY: designed the study, selected trials for inclusion, extracted data from. papers, appraised the quality of studies, carried out analysis, interpretation of the data, draft the manuscript, and approved the final manuscript as submitted. L-LZ: designed the study, selected trials for inclusion, appraised the quality of studies, interpretation of the data, draft the manuscript, and approved the final manuscript as submitted. L-NZ: undertook searches, selected trials for inclusion, extracted data from papers and appraised the quality of papers, commented the manuscript, and approved the final manuscript as submitted. YL: undertook searches, selected trials for inclusion, extracted data from papers and appraised the quality of papers, commented the manuscript, and approved the final manuscript as submitted. LH: undertook searches, extracted data from papers, entered data into SPSS and checked the data, commented the manuscript, and approved the final manuscript as submitted. Y-ZL: undertook searches, extracted data from papers, entered data into SPSS, checked the data and commented the manuscript, and approved the final manuscript as submitted.
About this article
Cite this article
Yang, CS., Zhang, LL., Zeng, LN. et al. 10-year trend in quantity and quality of pediatric randomized controlled trials published in mainland China: 2002–2011. BMC Pediatr 13, 113 (2013). https://doi.org/10.1186/1471-2431-13-113
- Randomized controlled trials
- Quality assessment