Randomized controlled trials in pediatric complementary and alternative medicine: Where can they be found?
© Sampson et al; licensee BioMed Central Ltd. 2003
Received: 28 May 2002
Accepted: 14 February 2003
Published: 14 February 2003
The safety and effectiveness of CAM interventions are of great relevance to pediatric health care providers. The objective of this study is to identify sources of reported randomized controlled trials (RCTs) in the field of pediatric complementary and alternative medicine (CAM).
Reports of RCTs were identified by searching Medline and 12 additional bibliographic databases and by reviewing the reference lists of previously identified pediatric CAM systematic reviews.
We identified 908 reports of RCTs that included children under 18 and investigated a CAM therapy. Since 1965, there has been a steady growth in the number of these trials that are being published. The four journals that published the most reported RCTs are The American Journal of Clinical Nutrition, Pediatrics, Journal of Pediatrics, and Lancet. Medline, CAB Health, and Embase were the best database sources for identifying these studies; they indexed 93.2%, 58.4% and 42.2 % respectively of the journals publishing reports of pediatric CAM RCTs.
Those working or interested in the field of pediatric CAM should routinely search Medline, CAB Health and Embase for literature in the field. The four core journals identified above should be included in their collection.
Keywordsalternative medicine bibliometrics information services medical informatics applications pediatrics, evidence-based medicine
Complementary and alternative medicine (CAM) is defined as a broad domain of healing resources that encompasses all health systems, modalities and practices and their accompanying theories and beliefs, other than those intrinsic to the politically dominant health system of a particular society or culture in a given historical period .
The safety and effectiveness of CAM interventions are of great relevance to pediatric health care providers. Two recent surveys indicate a use of pediatrics CAM in the United States, finding a 12% and 21% overall use, respectively [2, 3]. Factors such as ongoing medical problems and parents' use of CAM are associated with pediatric use [3, 4]. Ideally, evidence-based practitioners can approach children and their guardians regarding their use of CAM and on a case-by-case basis to help identify relevant evidence regarding therapies that patients use or wish to consider.
Such discussions are likely to be more rewarding when practitioners and families can advocate for the use of CAM therapies for which there is stronger evidence, such as from the results of randomized trials of effective herbal therapies . Similarly, interventions for which there is little of no evidence might be best avoided .
We undertook a program of research to discover whether such evidence, in the form of published systematic reviews and randomized controlled trials involving children, existed [7, 8]. Based on the large number of randomized controlled trials we did find, we identified a need for a guide for evidence-based practitioners who wish to determine whether sound evidence exists.
The objective of the research presented here was to identify where health care practitioners can locate reported RCTs of pediatric CAM; specifically, in which journals these reports are published and through which databases those reports may be identified.
Identification of Candidate Studies
Databases, date ranges, and strategies used.
Date Range Searched
Medline & Pre-Medline
Allied and Complementary Medicine (AMED)
Manual, Alternative and Natural Therapy (MANTIS)
The Cochrane Library CCTR
Update Software (CD-ROM)
Issue 1, 2001
NCCAM (now incorporated into PubMed)
IBIDS – NIH Office of Dietary Supplements
Studies published in languages other than English were included if: (1) the article was written in a language we were able to read (German, French, and Spanish); or (2) we were able to ascertain, from the English abstract, whether the study met the inclusion criteria. Only journal-published articles were considered. Theses, conference proceedings, book chapters, and other grey literature were not considered.
A post hoc analysis was undertaken to assess whether the apparent decline in the number of reports of trials in complementary and alternative medicine in the final years of this study was due to a delay in the indexing of these articles. For this analysis, the number of Medline records retrieved by search strategy 1 was broken down by year, ending with September Week 3 2002. Simple counts were taken – the individual items were not evaluated against the study inclusion criteria.
For each eligible publication, the date of publication, authors, language of publication, and journal name were extracted from the Reference Manager database or from inspection of the article. We consulted the reference material for each of the 13 databases searched to determine whether or not the database indexed each of the journals that had published one or more eligible reports of RCTs.
A total of 908 reports of RCTs met the criteria for inclusion in the study.
Year of publication
92.2% of reports of RCTs published in this field are written by more than one author. The largest number of authors for a single paper was 25 , while almost half of all papers (49.3%) have four or fewer authors. The five most prolific authors have published at least 15 reports of RCTs each. The vast majority (57.8%) of all authors cited in the 908 studies authored a single paper and 70% of authors published four or fewer CAM trials.
Language of Publication
Language of Publication
N of RCT reports
Danish, Hungarian, Norwegian, Slovak, Swedish
The reports of RCTs originated from 308 different journals. These journals were ranked by the number of reports of RCTs published. As a method to examine the dispersion of the literature, we divided the journals into four zones, each representing approximately one quarter of the articles published.
Journals in Zones 1 & 2 and the corresponding number of pediatric CAM RCT reports identified.
No. of RCT reports
No. of Journals
217 (23.9 %)
4 (1.3 %)
American Journal of Clinical Nutrition
Journal of Pediatrics
246 (27.1 %)
24 (7.8 %)
Journal of Pediatric Gastroenterology and Nutrition
Journal of Nutrition
European Journal of Clinical Nutrition
Archives of Disease in Childhood
New England Journal of Medicine
Chung-Kuo Chung Hsi i Chieh Ho Tsa Chih
British Journal of Nutrition
Journal of Tropical Pediatrics
Transactions of the Royal Society of Tropical Medicine & Hygiene
Archives of Disease in Childhood (Fetal and Neonatal edition)
Developmental Medicine & child Neurology
European Journal of Pediatrics
Journal of Pediatric Psychology
Br Med J (Clin Res Ed)
Journal of Manipulative & Physiological Therapeutics
Journal of the American College of Nutrition
Pediatric Infectious Disease Journal
(individual journal names not listed)
261 (28.7 %)
96 (31.2 %)
(individual journal names not listed)
184 (20.3 %)
184 (59.7 %)
Total number & percentage of journals indexed by database
N of Journals Indexed
% of total
Incremental n of journals indexed1
Cumulative % of journals indexed
N of zone 1 & 2 journals indexed (of 28)
Cumulative % of trials accounted for2
We identified over 900 reports of RCTs using a CAM intervention in children. We found a steady growth in the literature. We found fewer articles in most recent years, likely due to a lag in indexing. There was an apparent decline in the frequency of reporting from 1998 to 2001. We expected that the indexing of reports of RCTs published during this period would not have been completed by the time we conducted our research in 2001, but post hoc examination of the year by year number of items retrieved from Medline by search strategy 1 shows that some articles in this area can take more than one year to be indexed in bibliographic databases. Those wishing to obtain all journal articles describing randomized controlled trials of pediatric complementary and alternative medicine may wish to consider hand-searching the most recent years of key journals unless it can be determined that recent issues have been indexed by the bibliographic databases.
Our results indicate that most reports (93%) of pediatric CAM RCTs are published in English. This finding has been reported elsewhere , although not specifically for pediatrics. Although we had to exclude relatively few trials (n = 41) on the basis of language, and the databases we used provide content for journals published in many languages besides English, it is possible that we missed important clusters of non-English reports. In particular, Tang et al.  describe a large number of RCTs of traditional Chinese medicine conducted in China, (approximately 7500 published prior to 1997) as being for the most part inaccessible to Western doctors. However, the journal cited by Tang as having published ten times as many RCTs as any other they examined (Zhongguo Zhong Xi Yi Jie He Za/ Chinese Journal of Integrated Traditional and Western Medicine) is indexed in Medline with entries dating back to 1992. Of 1568 articles from that journal indexed, 284 are tagged as randomized controlled trials in Medline. Of these, only 87 relate to childhood age groups, and all of these have abstracts, thus would be have been picked up by our search and eligible for inclusion in our study. Still, Tang's review underscores the risks of over-reliance on database searching to identify all trials in a given area. For those wishing to search exhaustively, there is potentially great value to involving subject experts who can identify the locations of specialty literatures, such as the literature reporting trials of traditional Chinese medicine described by Tang et al.
The 908 reports were identified in more than 300 journals, yet almost a quarter (24%) of them were found in just four journals (American Journal of Clinical Nutrition, Pediatrics, Journal of Pediatrics, and The Lancet). By expanding the core journal list to 28 journals, it is possible to account for slightly more than half of all reports of pediatric CAM RCTs we found. These results suggest that most reports of randomized controlled trials of CAM are 'mainstream' medical journals. This finding is similar to that reported by Pittler and colleagues  who investigated the relationship between the statistical directions of CAM RCT results and where they were likely to be published. They found that mainstream medical journals with an impact factor greater or equal to 1 published an equal number of CAM trials with positive and negative results, while lower impact factor journals published more CAM trials with positive findings.
In terms of database coverage, Medline provided the best coverage of the Zone 1 and 2 journals, indexing all 28. Embase and CAB Health indexed 85.7% and 92.3% respectively. CINAHL and AMED indexed relatively few of the journals in Zones 1 and 2, (25% and 14% respectively) but they did provide unique material not available from the other databases. Together, these five databases gave complete coverage of all journals and reports of RCTs identified in this study. Two other databases, CCTR and IBIDS, provided relatively good coverage of these first two zones (65.3% and 57.1%), but because of overlapping coverage with other databases, provided no unique coverage.
In terms of database selection, a search of Medline alone could potentially identify 97.7% of these reports and a search combining Medline, Embase and CAB Health could achieve 99.4% retrieval, assuming a comprehensive search strategy is used. Given the recent development of the CCTR we were impressed with its coverage although its unique contribution is only marginal in terms of the cumulative increase of identified reports of RCTs resulting from searching this source. In an effort to further enhance the CCTR's coverage we have forwarded the database from this study to the Cochrane Collaboration's CAM field so that the information can be added to the CCTR.
Searching Medline, along with any two of Embase, CCTR, or CAB Health, would appear to constitute a reasonable approach for those wishing to search this topic comprehensively. More exhaustive searching, of the type undertaken for a systematic review, would ideally be supplemented by CINAHL and AMED, as well as selective hand searching and other traditional bibliographic techniques .
Those wanting to keep abreast of the latest pediatric CAM reports of RCTs by scanning journals as they are published, may be able to maximize their time by focusing their attention on four journals. Information specialists are likely to be able to track most new evidence by searching three databases.
Shortly after we completed the retrieval work for this project, the National Library of Medicine introduced a complementary medicine subset to its Medline database . Searches can be limited to this subset through PubMed (available at http://www.pubmed.org, and this feature is likely to become available through database vendors. This may prove to be a useful tool for clinicians wishing quick access to some of the best literature in this area. PubMed includes Medline, is available at no cost to those with Internet access and can be searched without complex syntax. As a case study, we entered the word random* in the search box, applied the limits of age "All Child: 0–18 years" and subset "Complementary Medicine" and publication type of "Randomized Controlled Trial" retrieved 1858 citations. A clinician replacing the search string random* with a subject term massage would find 56 citations as a starting point, the term chiropract* with the same limits would yield 29, st johns wort or st john's wort would yield 8 reports and herbal would yield 140 citations.
A limitation of this study is that our figures on the cumulative percent of trials accounted for (Table 4) assume that all trials published in that journal were indexed in the database. The actual indexing rate may be less than 100%, and the indexing status of each report of an RCT in each database was not verified in this study. In addition, researchers should be aware that indexing practices and search capabilities available with particular databases vary, thus there is some advantage to building redundancy in coverage into a search strategy that has high recall as an objective.
It should be noted that the Cochrane Controlled Trials Registry is a special case among the databases we searched. As its name suggests, it is a database of controlled trials, including randomized controlled trials, and is constructed from trials found in other databases and identified through hand searching of journals [19, 20]. At the time our searches were conducted, it was searchable only through its own proprietary interfaces. It has subsequently become available through at least one database aggregator, Ovid. Had such an interface been available at the time of our searching, we would have searched it using search strategy 1. This could potentially have increased its unique contribution beyond what is seen in Table 4. In a recent study in which known item searching techniques were used to determine if reports of randomized controlled were indexed in various databases, the CCTR performed extremely well, and was identified by the authors as a valuable but underused source for meta-analysts .
We used a strict definition of CAM, following the Registry Guidelines adopted by the Cochrane Complementary Medicine Field Registry of Randomized Controlled Trials, thereby excluding some psychological interventions that might be considered CAM by some (such as cognitive therapy) while including others (such as biofeedback). Our view is that some psychological interventions, such as cognitive behavior therapy, might well be considered a 'standard of care' in many Canadian and American settings.
We did not specifically follow a mega-dose definition for classifying nutritional therapies as CAM or conventional medicine. This may have overstated the contribution of the American Journal of Clinical Nutrition. We attempted to follow the Registry Guidelines for the Cochrane Complementary Medicine Field Registry of Randomized Controlled Trials but erred on the side of over-inclusion of therapies as CAM if the author's self-classification was not clear from the title, abstract and indexing information.
It is clear from this study that there is a large body of evidence from randomized controlled trials in pediatric complementary and alternative medicine. The quality of that literature is also relevant [22–24], although a recent paper suggests that quality measures may be less important than previously reported in obtaining unbiased estimates of intervention effectiveness in meta-analysis . Schulz and colleagues reviewed 250 reports of RCTs and found that those with inadequate allocation concealment, compared to reports in which this information was adequately reported, exaggerated the estimates of an intervention's effectiveness by 30%, on average. Recently, Moss and colleagues  reported that in a review of 134 pediatric surgical RCTs, less than half of the trials (46%) reported any details of the method of randomization used. Tang et al. notes that although improving, methodological problems remain in the large body of trials of traditional Chinese medicine . We previously reported on the quality characteristics of a subset of the trials found in this project .
RCT evidence in pediatric CAM is being published in a substantial number of journals. This study identifies the primary sources of reports of RCTs in the field of pediatric CAM-those journals found in Zone 1. These journals are a necessary core for any collection in this field. The journals in Zone 2 are also important sources of evidence, and are also recommended resources to be included in a pediatric CAM collection. Journals in Zone 3 would be a useful addition for those with larger collection budgets, and are good resources for those needing a comprehensive collection. The journals within Zone 4 contain only one report of an RCT each identified during the time period studied. While not prime candidates for a collection focusing on complementary and alternative medicine, they should be considered by systematic reviewers, who may identify potentially relevant studies from them through the relevant bibliographic databases. This study also indicates that those seeking pediatric CAM evidence require access to at least three indexing databases. These are Medline, Embase, and CAB Health.
List of abbreviations used
– Randomized Controlled Trial
– Complementary and Alternative Medicine
– Cumulative Index to Nursing and Allied Health
– Allied and Complementary Medicine
– The Cochrane Controlled Trials Register
– International Bibliographic Information on Dietary Supplements
– Manual Alternative and Natural Therapy
– The National Center for Complementary and Alternative Medicine
–AGRICultural OnLine Access
We thank Lynn McCleary for her helpful editorial advice.
- Panel on Definition and Description CRMCA1: Defining and describing complementary and alternative medicine. Alternative Therapies. 1997, 3: 49-57.Google Scholar
- Ottolini MC, Hamburger EK, Loprieato JO, Coleman RH, Sachs HC, Madden R, et al: Complementary and alternative medicine use among children in the Washington, DC area. Ambul Pediatr. 2001, 1: 122-125.View ArticlePubMedGoogle Scholar
- Sawni-Sikand A, Schubiner H, Thomas RL: Use of complementary/alternative therapies among children in primary care pediatrics. Ambul Pediatr. 2002, 2: 99-103.View ArticlePubMedGoogle Scholar
- Heuschkel R, Afzal N, Wuerth A, Zurakowski D, Leichtner A, Kemper K, et al: Complementary medicine use in children and young adults with inflammatory bowel disease. Am J Gastroenterol. 2002, 97: 382-388. 10.1016/S0002-9270(01)04036-9.View ArticlePubMedGoogle Scholar
- Ernst E: Herbal medicines: where is the evidence?. BMJ. 2000, 321: 395-396. 10.1136/bmj.321.7258.395.View ArticlePubMedPubMed CentralGoogle Scholar
- Complementary medicine: time for critical engagement. Lancet. 2000, 356: 2023-10.1016/S0140-6736(00)03392-4.
- Moher D, Soeken K, Sampson M, Ben Porat L, Berman B: Assessing the quality of reports of systematic reviews in pediatric complementary and alternative medicine. BMC Pediatr. 2002, 2: 3-10.1186/1471-2431-2-3.View ArticlePubMedPubMed CentralGoogle Scholar
- Moher D, Sampson M, Campbell K, Beckner W, Lepage L, Gaboury I, et al: Assessing the quality of reports of randomized trials in pediatric complementary and alternative medicine. BMC Pediatr. 2002, 2: 2-10.1186/1471-2431-2-2.View ArticlePubMedPubMed CentralGoogle Scholar
- The Cochrane Library. Oxford, Update Software. 2001, 1
- McKibbon A: PDQ Evidence-based principles and practice. Hamilton: BC Decker Inc. 1998Google Scholar
- Cochrane Complementary Medicine Field Registry Guidelines. 2001
- Landis JR, Koch GG, Freeman JL, Freeman DHJ, Lehnen RC: A general methodology for the analysis of experiments with repeated measurement of categorical data. Biometrics. 1977, 33: 138-158.Google Scholar
- Visalli N, Cavallo MG, Signore A, Baroni MG, Buzzetti R, Fioriti E, et al: A multi-centre randomized trial of two different doses of nicotinamide in patients with recent-onset type 1 diabetes (the IMDIAB VI). Diabetes Metab Res Rev. 1999, 15: 181-185. 10.1002/(SICI)1520-7560(199905/06)15:3<181::AID-DMRR31>3.3.CO;2-8.View ArticlePubMedGoogle Scholar
- Moher D, Pham , Klassen TP, Schulz KF, Berlin JA, Jadad AR, et al: What contributions do languages other than English make on the results of meta-analyses?. J Clin Epidemiol. 2000, 53: 964-972. 10.1016/S0895-4356(00)00188-8.View ArticlePubMedGoogle Scholar
- Tang JL, Zhan SY, Ernst E: Review of randomised controlled trials of traditional Chinese medicine. BMJ. 1999, 319: 160-161.View ArticlePubMedPubMed CentralGoogle Scholar
- Pittler MH, Abbot NC, Harkness EF, Ernst E: Location bias in controlled clinical trials of complementary/alternative therapies. J Clin Epidemiol. 2000, 53: 485-489. 10.1016/S0895-4356(99)00220-6.View ArticlePubMedGoogle Scholar
- Counsell C: Formulating questions and locating primary studies for inclusion in systematic reviews. In Systematic reviews: Synthesis of best evidence for health care decisions. Edited by: Mulrow C, Cook D. 1998, Philadelphia: American College of PhysiciansGoogle Scholar
- Nahim AM: Complementary medicine – new PubMed subset. NLM Technical Bulletin. 2001, 318: e7-Google Scholar
- The Cochrane Collaboration Handbook 4.1.1. In The Cochrane Library. Edited by: Clarke M, Oxman AD. 2000, Oxford: Update Software, 1
- Avenell A, Handoll HHG, Grant AM: Lessons for search strategies from a systematic review, in The Cochrane Library, of nutritional supplementation trials in patients after hip fracture. Am J Clin Nutr. 2001, 73: 505-510.PubMedGoogle Scholar
- Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, et al: Should meta-analysts search Embase in addition to Medline?. Journal of Clinical Epidemiology. 2003.Google Scholar
- Kjaergard LL, Villumsen J, Gluud C: Quality of randomised clinical trials affects estimates of intervention efficacy. 7th Annual Cochrane Colloquium Abstracts, in Rome. October 1999
- Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al: Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?. Lancet. 1998, 352: 609-613. 10.1016/S0140-6736(98)01085-X.View ArticlePubMedGoogle Scholar
- Schulz KF, Chalmers I, Hayes RJ, Altman DG: Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995, 273: 408-412. 10.1001/jama.273.5.408.View ArticlePubMedGoogle Scholar
- Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, et al: Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA. 2002, 287: 2973-2982. 10.1001/jama.287.22.2973.View ArticlePubMedGoogle Scholar
- Moss RL, Henry MC, Dimmitt RA, Rangel S, Geraghty N, Skarsgard ED: The role of prospective randomized clinical trials in pediatric surgery: state of the art?. J Pediatr Surg. 2001, 36: 1182-1186. 10.1053/jpsu.2001.25749.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2431/3/1/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.