Was the treatment pressure for the control group 1.3 atmospheres gauge pressure or 1.3 atmospheres absolute? From the wording of the article it appears as if it was 1.3 atmospheres absolute, or a depth equivalent of 10 feet of sea water. If this is the case, then the treatment group received an O2 partial pressure of about 0.31 atmospheres absolute, or a surface equivalent of 31% O2. If the treatment pressure was 1.3 atmoshperes gauge (2.3 atmospheres absolute, or a depth equivalent of 43 feet of sea water), then the treatment group received an O2 partial pressure of 0.55 atmospheres absolute, or a surface equivalent of 55% oxygen. Both of these can be delivered to a patient without the use of a hyperbaric chamber. Was any consideration given to using a second control group which received surface oxygen at a partial pressure equivalent to that delivered to the hyperbaric treatment group?
Competing interests
None declared
Hyperbaric Oxygen
William Anderson, Retired
1 April 2009
24% Oxygen at 1.3 atmospheres is equivalent to 31.2% at 1 atmosphere. This is not by any normal usage hyperbaric oxygen. The same Oxygen levels could have been achieved at less distress to the children and less cost with a simple face mask or even in a small room. If the pressure as opposed to the oxygen is the dimension of interest then 1.3 atmospheres at 21% could have been compared with 27.3% at ambient pressure. Autism is a terrible affliction and we owe the sufferers our best efforts.
Competing interests
A commitment to offering ill people only interventions with some plausible likelihood of benefit.
Two Small Points
James (Undisclosed), Autism Street Blog
1 April 2009
This article is quite interesting. While on the surface it seems to lend some scientfic validation for a possible treatment of autism, it may raise questions - about validity of the results based on the methodology employed, and possible clinical and economic implications of the therapy described.
Firstly, the stated treatment pressure (1.3ATM) would seem highly unlikely to be uniformly achievable across all study locations with the equipment that appears to have been used.
As described in the section titled, "Interventions":
"These procedures included covering control switches, inflating and deflating the chambers to simulate pressure changes, and masking the sounds from the chambers."
The use of "inflatable" monoplace chambers, sometimes known as "portable" hyperbaric chambers, is a clear indication that the actual total pressures (and results of this study) would have been affected by the ambient air pressures at the times and locations of treatment.
It appears that the study sites vary from low lying cities such as Edison, NJ (115'AMSL) and Melbourne, FL(33'AMSL) to cities of modest elevation such as Lynchburg, VA (938'AMSL) and Phoenix, AZ (1135'AMSL).
Even if it is assumed that the chambers at the different sites operate with similar precision, and it is also assumed that they fully pressurize to 4.4 PSI (.3 ATM), the actual gauge pressure that would be required to deliver the full stated 1.3 ATM at sea-level for an inflatable chamber (1 ATM of ambient pressure + .3 ATM added pressure = 1.3 ATM, or more accurately, 1.3 ATA), there could be a 10% or more deviation in added treatment pressure between several sites due to elevation alone.
Local atmospheric pressure is typically reported as sea-level pressure1 for aviation, and weather maps, etc., but the actual station pressure is affected by the elevation. As an example, calculated station pressure (the ambient air pressure) at Melbourne, Florida would be approximately 14.68 PSI, and with 4.4 PSI of added treatment pressure (.3 ATM), the actual total pressure would be the stated 1.3 ATM (more accurately 1.3 ATA). The same treatment in Phoenix, Arizona would be quite different. Calculated station pressure (the ambient air pressure) there, would be approximately 14.13 PSI, and with 4.4 PSI of added treatment pressure (.3 ATM), the actual total pressure would be only 1.26 ATA.
With a theoretical maximum pressure increase of 4.4 PSI (.3 ATM) above the ambient pressure, a potential error of .04 ATM (by assuming ambient sea-level pressure exists at elevation when it probably does not) would mean that 13% of the expected pressure increase above 1 ATM was never delivered. This may not be insignificant at such low added pressures in terms of total oxygen delivery. If the added treatment pressure of the inflatable chambers is less than the full 4.4 PSI, the error grows. Furthermore, the same problems with calculating actual treatment pressures will apply to the "placebo" control group.
Secondly, the authors of this article should be commended for fully disclosing their competing interests (derivation of revenue from hyperbaric treatment in their practices), in light of what the results of this study may mean if replicated.
The oxygen delivered at 24% FIO2 and 1.3 ATA via a portable hyperbaric chamber can be easily matched and even surpassed with simple oxygen therapy and no hyperbaric chamber at all (e.g. a simple face mask delivering 35-55% FIO2 at 5-8 lpm2 @ 1 ATA). While the authors cite an in vitro study for a hypothetical effect of pressure alone, the notion of in vivo and clinical significance in autism from such a small pressure change in and of itself doesn't seem to have much (if any) scientific support. All things considered, the results of this study suggest that simple oxygen therapy (at a fraction of the cost of hyperbaric oxygen therapy with inflatable chambers), may be an effective autism treatment.
2National Institutes of Health - Critical Care Medicine Department, Critical Care Therapy and Respiratory Care Section Category: Clinical, Section: Medicinal Gas Therapy, Title: Oxygen Therapy Procedure, Policy: 01, Revised: 02/02
Competing interests
Financial Competing Interests: None
Personal Compteting Interests: I'm the father of an autistic child, and author of a non-commercial blog that often examines autism treatment claims with scientific skepticism.
Normobaric oxygen vs Hyperbaric Oxygen
Kenneth Stoller, International Hyperbaric Medical Association
6 April 2009
Oxygen dosing (which is modulated by pressure far more than absolute concentration)is what provides the signaling at a cellular level, including the subcellular level in the mitochondria, to facilitate the changes observed using a hyperbaric environment.
No where was this made more clear than in a 163 patient prospective study of carbon monoxide poisoning patients who were given different doses of supplemental normobaric oxygen. Six week post-treatment cognitive sequelae were unchanged - contrast that to the results when hyperbaric oxygen is used. (Weaver et al in reply to Scheinkestal et al: The role of hyperbaric oxygen in carbon monoxide poisoning. Emerg Med Australa 2004;16;394-399).
The gas laws and hyperbaric medicine are not taught in medical school, so it is unfortunate that these questions have to come up, because there is a fundamental lack of understanding about what oxygen under pressure does. This problem is far greater than treating autistic children - this endemic lack of understanding about what hyperbaric oxygen does and when to use it has kept hyperbaric chambers out of Emergency Departments everywhere when there should be a chamber in every trauma center.
In the case of autistic children and children with CP, the therapeutic window, or that pressure that will effect DNA signaling, appears to be very low - in many cases less than 1.3 ATA. That has been my experience with dozens of children so affected. However that is not the appropriate pressure (oxygen dose) for treating other conditions.
Oxygen dose is poorly understood in hyperbaric medicine and even less understood outside of the field, again there is a lot of misapplication. Breathing in normobaric oxygen, regardless of the concentration is not comparable to using oxygen under pressure.
Competing interests
President of the International Hyperbaric Medical Association
Not just Oxygen?
Edward Fogarty, M.D., UND SOM SW Campus
6 April 2009
Hopefully this study will be repeated in a multi-center trial in academic medical centers as it does seem promising, with some added neurofunctional imaging such as SPECT, PET or fMRI with MRS. The difficulties of shamming or placebo grouping this intervention seems to have plagued the idea since its first inceptions for medical therapy. I simply wonder on the basis of having read the following article whether too much focus has been placed on oxygen as the sole therapeutic effector in this treatment modality?
Stem cell mobilization by hyperbaric oxygen. Thom SR, Bhopale VM, Velazquez OC, Goldstein LJ, Thom LH, Buerk DG. Am J Physiol Heart Circ Physiol. 2006 Apr;290(4):H1378-86. Epub 2005 Nov 18. PMID: 16299259 [PubMed - indexed for MEDLINE]
Competing interests
Child with an autistic disorder and research/academic interests in HBOT.
hyperbaric vs. normobaric oxygen (response)
Eric Hexdall, Duke University Hospital
16 April 2009
Dr. Stoller's prolific writing and his efforts toward educating the public on hyperbaric medicine are respectfully acknowledged. I would disagree with his implication that the mechanism of action of hyperbaric oxygen is poorly understood in the hyperbaric community. Granted, some of the biomechanics of oxygen at the cellular and subcellular level remain unclear. However, Dalton's Law, Henry's Law and the alveolar gas equation are familiar to most of us, and they all support the contention that 0.31 ATA of oxygen is 0.31 ATA of oxygen no matter what the delivery method. Perhaps there is an as-yet-unknown synergistic effect of slight increases of ambient pressure on "normobaric" oxygen pressures, i.e. partial pressures of O2 that are less than or equal to 1.0 ATA (as this study suggests). If so, this effect needs to be carefully studied and documented, not merely taken for granted. Legitimate scientific questions will arise from the medical community at large, and we must be prepared to answer them if we are to be perceived as credible. We need to control for all variables, not just those that appear to prove our postulations.
Competing interests
None declared
Gas Laws Not Mysterious - They Apply Here
James (Undisclosed), Autism Street Blog
16 April 2009
Comments by Mr. Stoller indicate that oxygen dosing is more modulated by pressure than absolute concentration. While that may be especially true when concentrations of O2 (FIO2) approach 100%, the statement does not apply with respect to this study. In fact, inflatable chambers have built-in pressure relief valves that, unless modified (illegally), limit the total pressure at about 4psi above ambient atmospheric pressure. Without the ability to add additional pressure above this very low maximum, the ability to "modulate" pressure, except towards even less pressure (and lower oxygen dose), does not even exist. Citation of Weaver et al. fails to add relevance with respect to this study (or autism) - The Weaver et al. treatment group seems to have received real hyperbaric oxygen therapy (100% 02 at 3 and 2 ATA).
In response to Mr. Stoller's comments regarding hyperbaric medicine in general, the gas laws are taught in U.S. medical schools, in the sections on pulmonary physiology and again during blood physiology. Hyperbaric oxygen therapy is also discussed during these sections.
Mr. Stoller writes, "Breathing in normobaric oxygen, regardless of the concentration is not comparable to using oxygen under pressure."
When dealing with added pressures that are so low (like the .23-.27 ATM in this study), and FIO2 of 24%, they are quite probably very comparable.
Unless there is a known and proven clinical effect of such low pressure alone for autism (which doesn't appear likely at this point), the only factor that provides hypothetical effect is the increase in partial pressure of oxygen. Cellular physiology is dependent on oxygen partial pressure, which is the product of the total pressure and the fraction (percentage) of oxygen inspired. Total pressure - apart from its impact on partial pressure - has negligible effect on oxygen delivery.
One additional question for the original authors: Can you provide more detail about how the multiple significance tests might have corrected for?
Competing interests
Financial Competing Interests: None
Personal Competing Interests: I'm the father of an autistic child, and author of a non-commercial blog that often examines autism treatment claims with scientific skepticism.
correction
Eric Hexdall, Duke University Hospital
17 April 2009
In my first comment, I mistakenly asked if the treatment pressure for the CONTROL group was 1.3 atmospheres absolute or 1.3 atmospheres gauge. My question was about the TREATMENT group. I regret any misunderstanding that may have arisen as a result.
Competing interests
None declared
Results of the Study are NOT Positive as Claimed
Mark Mintz, The Center for Neurological and Neurodevelopmental Health
7 May 2009
Dear Editor/Authors:
We read with interest the recently published article by Rossignol et. al.(1) concerning the use of hyperbaric oxygen treatment (HBOT) in children with Pervasive Developmental Disorder-Autistic Disorder. As the authors have rightly pointed out, there is a need for more evidenced-based treatments for this distressing disorder, and they should be commended for undertaking a controlled clinical trial. But we are concerned that the evidence as presented by these authors does not provide support for the use of HBOT in this population despite the authors’ conclusions and claims to the contrary. In particular, we believe there are significant methodological and statistical flaws in the design, implementation and interpretation of the results of this clinical trial, which have rendered the conclusions of the authors faulty and mistaken. In particular, we invite the authors to respond to the following:
1. In the Methods section, it is stated “…At the end of the study, all children assigned to the control group were offered 40 hyperbaric treatments at the treatment pressure (1.3 atm and 24% oxygen) if the parents desired (all parents chose this option; treatments were provided without charge)...” Essentially, this is a break of the blind, as the investigators and families would know who were in the control group prior to the completion and analysis of the study for all participants. If an open label extension was to be offered, all participants enrolled in the double blind treatment phase would need to be offered the open label treatment to maintain the integrity of the double blind. 2. It is legitimate to provide an analysis of the intent-to-treat (ITT) group as was done in this paper, but this should be done in conjunction with an analysis of the per-protocol (PP) group to provide valid analyses and conclusions. This is particularly necessary in this study, as a primary endpoint measure of the study, the Autism Treatment Evaluation Checklist (ATEC), could not be done for approximately 30% of the total study group as a result of a protocol violation, as was stated in the Methods “…Due to an administration error, the baseline ATEC was not performed at one of the study centers, and thus data was available for analysis for 23 children in the treatment group and 21 children in the control group...” Additionally, it is concerning that there was one site of six that accounted for 30% of recruitment, and yet could not properly perform the protocol procedures for collecting primary endpoint data. What was the integrity of the remainder of the data from that site? Why was there not a better distribution of enrollment across sites?
Thus, of the N = 62 subjects enrolled/randomized, only N = 44 had an ATEC done, with an additional seven dropouts, resulting in a PP group of N = 37. What were the results of the data for the PP group? How does the PP group compare to the ITT group? Thus, with a PP group of N = 37, it is likely that the PP group is not powered sufficiently, as the authors have stated a sample size of N = 43 would be necessary for an effect size of d = 0.44. 3. We have concerns over the statistical methods and presentation of the data. In Figures 2-4, there are no error bars, which if used would graphically display the lack of significant differences between the treatment and control groups. Figure 2 is very confusing: how can there be a representation of a change from baseline for the CGI-I, when it is a scale that is given only at the end of the treatment phase, as stated in the Methods section, and thus, no baseline for comparison? The data as presented indicate only a slight intergroup difference in the subjective Clinical Global Impressions-Improvement (CGI-I) as completed by the physicians, but not the families; one subscale of the ABC; and one subscale of the ATEC. Yet, there is a wide error margin for the scoring, and there is no Effect Size calculation, and given the small sample size, the Effect Size is likely insignificant. 4. In the listing for this clinical trial on www.clinicaltrials.gov, it is stated that there is a Blinded Physician for the CGI (no mention of a parent CGI), Blinded Therapist for the ABC, and Blinded Therapist for the ADOS. The ATEC is listed as being given by the parent, but it is not noted that the ATEC evaluator is blinded, and no mention of a physician or therapist ATEC rater. Yet, in the Methods section, it is stated that the ABC, CGI and ATEC were done by the parent or caretaker, the CGI by the physician, and no mention of any Therapists administering any of the scales. Why this contradiction? Were the scales done as a structured interview? Was there any prior training or inter-rater reliability analyses of the scales prior to site initiation to improve the validity and consistency of results? Why was “placebo” taken out from “placebo control group” in 2007? 5. In the opening Abstract, the authors conclude that there were significant improvements in “…overall functioning, receptive language, social interaction, eye contact, and sensory/cognitive awareness...” This is misleading, as the authors are referring to an intragroup analysis of the treatment group only. This is irrelevant to the design and power calculations of the study, which was designed to compare a treatment group to a control group. When analyzing the results of the comparative analyses, there is only a slight difference noted as described in paragraph #3 (above), with the additional confounder that the ATEC could be analyzed for only 60% of the ITT group. 6. In looking at the peer reviews of the paper, there were concerns about table 2 in the original submission, with Reviewer #2 having trepidation that there were too many variables being analyzed, which can lead to false statistical conclusions via the Bonferroni effect (the chance of finding a positive result is linearly proportional to the number of statistical correlations).(2) Why did the authors eliminate this table of data from the final publication, rather than allowing the reader to see all the data? 7. There is an extensive Discussion section that purports to provide a physiological basis for the allegedly “positive” results of this clinical trial, namely a reversal of “cerebral hypoperfusion” or suppression of “inflammatory” mechanisms. However, this clinical trial did not include any surrogate markers for assessing cerebral perfusion or immunological status, and thus, such an extensive discussion is more applicable to a review paper than a paper reporting results of a clinical trial with inconclusive results. 8. In their conclusions, the authors state that HBOT is “…feasible and not necessarily costly…” but have not provided any foundation for such a statement. It is the authors’ experience that patients who have tried HBOT have spent large sums of money not reimbursable by insurance. What is the average cost of 40 treatments over four weeks across the six participating centers?
The authors rightly conclude that it is necessary to await the results of other ongoing HBOT studies from other centers before there can be any statements that HBOT is efficacious or safe for children with Autism Spectrums Disorders. Thus, it is perplexing that the authors subsequently conclude “…in light of the positive results of this study…hyperbaric treatment appears to be a promising treatment for children with autism…” The results from this study do not support such a statement.
A critical reading and analysis of the data as presented by Rossignol et. al. has provided evidence that HBOT does not benefit children with Autism or Autism Spectrum Disorders, and at this point in time, HBOT is not a justified or evidence-based therapeutic intervention. Yet, the authors provide antithetical and misleading conclusions. Furthermore, this clinical trial has major methodological and statistical flaws and protocol violations that render the few questionable but clinically insignificant positive statistical correlations meaningless and inconsequential. Families of children with Autism Spectrum Disorders must deal with difficult decisions for weighing a large number of intensive treatment options that subject their loved ones to potential harm, as well as emotional and financial stress. It is important for the medical community to continue to explore new and promising therapies for this population. However, this study, despite its misleading conclusions, does not provide sufficient support or evidence for the use of HBOT for children with Autism Spectrum Disorders.
We await the authors’ response.
Mark Mintz, M.D. The Center for Neurological and Neurodevelopmental Health and the Clinical Research Center of New Jersey
Jonathan W. Mink, M.D., Ph.D University of Rochester School of Medicine and Dentistry
Max Wiznitzer, M.D. Rainbow Babies and Children’s Hospital
1. Rossignol DA, Rossignol LW, Smith S, et al: Hyperbaric treatment for children with autism: a multicenter, randomized, double-blind, controlled trial. BMC Pediatrics 2009, 9:21; doi:10.1186/1471-2431-9-21 2. Shaffer, J. P. "Multiple Hypothesis Testing." Ann. Rev. Psych. 46, 561-584, 1995.
Competing interests
No Financial Competing Interests
The Authors Respond: Results of the Study are Positive
Daniel Rossignol, International Child Development Resource Center
9 June 2009
We thank Drs. Mintz, Mink, and Wiznitzer for their comments on our article reporting our double-blind controlled study demonstrating the effectiveness of hyperbaric treatment in children with autism. We must point out fundamental errors in their review of our study, its methodology, and its data analyses, however, and clarify any misinterpretations of our findings.
1. With regard to the blinding procedure, only after all study data had been collected, taken to a central location, and analyzed were the control group patients unblinded and offered 40 hyperbaric treatments. These subsequent additional treatments were provided to the controls at a location remote from all study sites. Thus, the integrity of the double-blind procedure was kept completely intact throughout both the data collection and analysis stages. Consequently, there was not “a break of the blind” as the commentators contend.
2. An administrative error occurred at one site, which did not perform the ATEC at the beginning of the study. No other protocol violations occurred at that site or any other sites. Thus, the ABC, CGI scales, and all other study protocols were performed properly at all sites. Despite obtaining ATEC data for only 44 of the 56 children due to the administrative error, there was a significant change on the ATEC sensory/cognitive awareness subscale (p = 0.0367; effect size = 0.55) in the treatment group compared to the control group. The administration error did not affect the CGI or ABC scales, and data for these scales was collected on all children in the study. The site with this administration error enrolled 12 children in the study (19% of the children in the study; the distribution of enrollment across sites was fairly even). None of the 44 children who received an ATEC dropped out of the study and therefore the commentators have wrongly subtracted 7 dropouts from the 44, and thus the number of 37 is erroneous. Since 56 children were analyzed in the study, the ATEC was performed on 44/56 children (79%), who were analyzed in accordance with Figure 1. Because all 44 children were properly accounted for, the “per-protocol” analysis for the ATEC is the same as the intention-to-treat analysis reported in the article. Thus the per-protocol analysis was of sufficient power and effect size.
3. Prior to initiating hyperbaric treatment, each child was examined by a physician who observed and noted the child’s performance/behavior on the subscales of the CGI at the baseline exam. At the conclusion of the 40 treatments, the physicians repeated their assessments, completing their clinical global impression as compared to baseline. The resulting scores were then used for the statistical analysis (found on pages 7 and 8) in which the mean scores on the CGI between the treatment group and the control group were compared. Figure 2 visually represents the change in score from baseline and does not present any new data. The p-values listed in Figure 2 are the same as those found on pages 7 and 8 and were derived from an intergroup comparison (treatment group compared to control group). Contrary to the comments of Drs. Mintz, Mink, and Wiznitzer, these improvements were statistically significant. Please see the discussion of point #5 below for further comments.
There are no error bars on the figures, but there is no data reported in the figures that can not be obtained from the results section or the tables. Furthermore, all of the scores in the paper are reported with standard deviations, as found on pages 7-9 and the tables. The effect sizes were not reported in the article, but are listed here. The results are actually robust: for the physician CGI score for overall functioning (effect size = 1.0); for the parental CGI score for overall functioning (effect size = 0.62); and for the ATEC sensory/cognitive awareness subscale (effect size = 0.55).
4. The clinicaltrials.gov website is not intended to include intricate details of a study, but instead serves as a registry of studies. Registration of a study on that site is required by most journals for publication of a corresponding article. For this study, a blinded psychologist completed a pre- and post-ADOS scale and observed no significant change in autism diagnosis between the two groups (this data is found on page 13 of the article; the ADOS was also used to confirm the diagnosis of autistic disorder prior to enrollment in the study). The ATEC and ABC are parent-generated data gathering tools (i.e., see reference [1]), and parents were given instructions regarding their proper use. All of the evaluators for all of the scales in this study (parents, physicians, and psychologists) were blinded throughout the entire study. The term “placebo” was originally listed in the clinicaltrials.gov website. Most hyperbaric studies that are controlled have used 1.1 atm as a “placebo” (for example, see reference [2]). Since we provided some pressure in the control group, we came to the conclusion that this was not a true placebo but a “near-placebo” as described in the abstract and on page 2. However, given the example provided by other hyperbaric studies, we would have been justified to call the control pressure a “placebo” if we had desired.
5. It appears that comment #5 lies at the heart of the commentators’ assertion that this study was “NOT Positive as Claimed.” Careful readers will note a number of significant improvements in the treatment group versus the control group, and that all of the analyses for the improvements listed in the conclusion of the abstract (“overall functioning, receptive language, social interaction, eye contact, and sensory/cognitive awareness”) were performed using an intergroup analysis (comparing the treatment group to the control group) not “an intragroup analysis of the treatment group only” as the commentators state.
The abstract clearly and plainly states “After 40 sessions, mean physician CGI scores significantly improved in the treatment group compared to controls in overall functioning (p = 0.0008), receptive language (p < 0.0001), social interaction (p = 0.0473), and eye contact (p = 0.0102); 9/30 children (30%) in the treatment group were rated as "very much improved" or "much improved" compared to 2/26 (8%) of controls (p = 0.0471); 24/30 (80%) in the treatment group improved compared to 10/26 (38%) of controls (p = 0.0024).” This is clearly an intergroup analysis (which is also described on pages 7 and 8 of the study), and its results are statistically significant. There was also significant changes on the CGI as rated by parents when comparing scores of the treatment group to the control group, as described on pages 7 and 8 and as found in the abstract: “Mean parental CGI scores significantly improved in the treatment group compared to controls in overall functioning (p = 0.0336), receptive language (p = 0.0168), and eye contact (p = 0.0322).” Therefore, there were statistically significant improvements using an intergroup analysis, in opposition to what the commentators stated in this point. On the ABC, the abstract also gives results for the intergroup analysis: “In the treatment group compared to the control group, mean changes on the ABC total score and subscales were similar except a greater number of children improved in irritability (p = 0.0311).” Finally the change on the ATEC sensory/cognitive awareness subscale was significant and was performed using an intergroup analysis: “On the ATEC, sensory/cognitive awareness significantly improved (p = 0.0367) in the treatment group compared to the control group.” As described above, the ATEC was performed on 79% of the 56 children who were eligible for analysis, not 60% as the commentators state. Most notably, rather than a “slight difference” as alleged by the commentators, we measured statistically significant (intergroup) improvements in the treatment group when compared to the control group in these measures. Thus, the only reasonable conclusion is that the studied treatment was positive.
6. The scales we used for comparisons between groups in this study were based upon our two previous studies using hyperbaric treatment in children with autism. Thus, we used planned comparisons in the current study, making the statistical analysis stronger because it was hypothesis based. The Bonferroni correction is one method for correcting for inflation of the alpha when conducting multiple statistical tests. However, when the dependent measures are correlated, as commonly occurs in clinical trials, the Bonferroni correction becomes too conservative and is not appropriate. Since, in such cases, there is little agreement on the precise method for calculating a statistical correction, some investigators suggest using a global assessment measure rather than using a Bonferroni correction [3]. This is exactly what we did in the current study. Indeed, we used the CGI score to confirm change in overall functioning as compared to baseline. Furthermore, during the review process, Reviewer #2 indicated that he did not “feel adequately qualified to assess the statistics” and suggested a “review by a statistical expert.” Prior to publication, the journal obtained an independent statistical review of the study’s data, which confirmed our findings.
7. The discussion section is meant to discuss the implications of the results of a study and possible reasons for the outcome, and so the discussion in this article is traditional, expected, and entirely reasonable. As requested by reviewers, we added further material to the discussion section concerning the possible “physiological basis” for the improvements found in this study. As noted in our discussion, we could not be certain what effect the hyperbaric treatment had on markers of inflammation or hypoperfusion, since we did not measure markers of these particular parameters in this study.
8. The average cost for hyperbaric treatments using the parameters performed in this study varies depending on the location in the country, but a reasonable estimate is approximately $40-60 per 60 minute treatment. The commentators, some of whom in actuality do have competing financial interests in therapy-based services performed at their respective centers, make a valid point to discuss cost and benefit analysis. While we recognize the importance of speech, occupational, and applied behavioral analysis therapy, we contend that hyperbaric treatment may actually be less costly with respect to the benefits obtained. Of course, the cost of hyperbaric treatment should be considered in the risk, benefit, and cost analysis discussion between the family and clinician, and decisions made based on the circumstances of each individual patient.
We did not state or conclude that “it is necessary to await the results of other ongoing HBOT studies from other centers before there can be any statements that HBOT is efficacious or safe for children with Autism Spectrum Disorders.” The safety of this treatment (using the parameters in this study) is established given the results of this study and previously published studies in autism [4-6]. Furthermore, when performed properly, this treatment has been studied in children and proven to be safe at higher pressures and oxygen concentrations than those used in this trial [7]. Since this is a double-blind controlled trial with statistically significant results, the efficacy of hyperbaric treatment in autism is now more proven than not. On page 13, we did state that “further studies are needed by other investigators to confirm these findings” but that statement does not mean we think that hyperbaric treatment (at the parameters used in this study) is not safe or not efficacious. We realize that other studies will need to replicate these findings before most physicians will change their practice and treatment recommendations. Based upon our findings, and the previously published studies of hyperbaric treatment in children with autism, it is completely reasonable to conclude that this treatment is “promising.”
A close review of the comments by Drs. Mintz, Mink, and Wiznitzer demonstrates significant errors in their interpretation. To reiterate, the analyses finding significant improvements in this study (i.e., p < 0.05) clearly were performed using an intergroup analysis (treatment group compared to control group), not an intragroup analysis as the commentators allege.
Finally, we are very aware of the difficult decisions that families of children with autism need to make concerning possible treatment options, and, in fact, designed and performed this study to help provide further objective information upon which families could base such difficult decisions. We are confused as to why the commentators state, without support, that the treatment we have evaluated would demonstrate “harm” to the patient, as we are not aware of any evidence of harm that has come to a child using hyperbaric treatment within the parameters used in this study. In fact, this study and the previous published studies of hyperbaric treatment in children with autism (using the protocol in this study) have reported that this treatment is safe.
In conclusion, we maintain our position that hyperbaric treatment using the parameters in this study is a cost-effective and promising medical adjunctive intervention to ameliorate autism symptomatology.
Dan Rossignol Lanier Rossignol Scott Smith Cindy Schneider Anju Usman Jim Neubrander Eric Madren Gregg Hintz Barry Grushkin Elizabeth Mumper
References
1. McCracken JT, McGough J, Shah B, Cronin P, Hong D, Aman MG, Arnold LE, Lindsay R, Nash P, Hollway J et al: Risperidone in children with autism and serious behavioral problems. N Engl J Med 2002, 347(5):314-321.
2. Clarke RE, Tenorio LM, Hussey JR, Toklu AS, Cone DL, Hinojosa JG, Desai SP, Dominguez Parra L, Rodrigues SD, Long RJ et al: Hyperbaric oxygen treatment of chronic refractory radiation proctitis: a randomized and controlled double-blind crossover trial with long-term follow-up. Int J Radiat Oncol Biol Phys 2008, 72(1):134-143.
3. Feise RJ: Do multiple outcome measures require p-value adjustment? BMC Med Res Methodol 2002, 2:8.
4. Rossignol DA, Rossignol LW: Hyperbaric oxygen therapy may improve symptoms in autistic children. Med Hypotheses 2006, 67(2):216-228.
5. Rossignol DA, Rossignol LW, James SJ, Melnyk S, Mumper E: The effects of hyperbaric oxygen therapy on oxidative stress, inflammation, and symptoms in children with autism: an open-label pilot study. BMC Pediatr 2007, 7(1):36.
6. Chungpaibulpatana J, Sumpatanarax T, Thadakul N, Chantharatreerat C, Konkaew M, Aroonlimsawas M: Hyperbaric oxygen therapy in Thai autistic children. J Med Assoc Thai 2008, 91(8):1232-1238.
7. Ashamalla HL, Thom SR, Goldwein JW: Hyperbaric oxygen therapy for the treatment of radiation-induced sequelae in children. The University of Pennsylvania experience. Cancer 1996, 77(11):2407-2412.
Competing interests
Authors of original article
Cost-effective Oxygen Delivery
James (Undisclosed), Autism Street Blog
24 June 2009
Dr. Rossignol writes:
"We are confused as to why the commentators state, without support, that the treatment we have evaluated would demonstrate “harm” to the patient, as we are not aware of any evidence of harm that has come to a child using hyperbaric treatment within the parameters used in this study."
This confusion is understandable, but unfounded. A quick review of the commentators' actual words and context reveals that no such statement, that the treatment evaluated by Rossignol et al. would demonstrate harm, was made.
Dr. Rossignol also notes:
"In conclusion, we maintain our position that hyperbaric treatment using the parameters in this study is a cost-effective and promising medical adjunctive intervention to ameliorate autism symptomatology."
Monthly cost for two $40-$60 daily treatments (60 minutes each, 5 days a week)?
About $2,000
Monthly cost for identical oxygen delivery with simple O2 therapy for same treatment period?
About $200
Competing interests
Financial Competing Interests: None
Personal Competing Interests: I'm the father of an autistic child, and author of a non-commercial blog that often examines autism treatment claims with scientific skepticism.
Response to the Authors’ Response: Results of the study are NOT positive as claimed
Mark Mintz, The Center for Neurological and Neurodevelopmental Health
12 August 2009
We thank Drs. Rossingol et al. for responding to our posted comment about our concerns with their study’s methodologies, results and conclusions. The authors’ response has not changed our analyses and opinions, and in particular has not changed our conclusion: it is our opinion that the paper does not support the use of hyperbaric oxygen therapy (HBOT) for children with autism. In particular, we invite the authors to explain or comment on the following:
1. In the authors’ response, they have not clarified how a physician-completed Clinical Global Impression-Improvement scale (CGI-I) can be used at baseline and then again after 40 treatment sessions. [In the Methods section, it states that physicians completed the CGI-I after 40 treatment sessions, but in the authors’ response to us there is a claim that there was a baseline assessment as well.] The CGI-I is a scale of improvement, and thus, does not provide a baseline measure, providing only a subjective end-of-study “impression” of change over the course of the protocol. Thus, there cannot be an outcome measure stating there was “no change” at baseline, which is what the authors are using for their statistical comparison [Figure 2].
2. We have tried to recreate the statistical result of the primary outcome measure of the study: the physician-completed Clinical Global Impression-Improvement scale (CGI-I). The data presented under “Outcome Measures” in the text appears to provide for scores represented as a mean +/- standard error of measurement (SEM) for the Treatment Group 2.87 +/- 0.78. Using frequencies for the Control Group, a mean of 3.62 +/- 0.75 is derived. If one uses a two-tailed t-test, then the result is t=0.69 yielding a non-significant p-value of p=0.49. Thus, it is unclear to us how the authors were able to report a p-value of 0.0008. Did the authors use a mean +/- standard deviation, and if so, how was this derived? Again, how can there be a statistical outcome measure compared to “no change” at baseline: how does one surmise there is “no change” prior to any intervention?
3. Our concerns about harm were not specifically directed at HBOT, but more generally at the use of unproven therapies with scant pediatric safety data, causing families to direct limited resources towards unverified therapies, and creating tremendous burdens on family time in pursuing treatments that have not undergone more rigorous scientific scrutiny (many examples exist, but secretin is one recent example: an expensive therapy that became widely used based on scant data, but found to be no better than placebo in a number of subsequent rigorous scientific studies). Possibly HBOT does not have major side effects at the parameters used in this study (and as argued by others on the Comments page, at these parameters, HBOT is likely no better than non-pressurized supplemental oxygen at 23%), but it has sparked false expectations and hope among well-meaning and desperate families in search of a cure for their children, and as such, can contribute to harm in a number of ways, illustrated by the tragic death of an Italian child and grandmother as they sought HBOT in Florida. [http://cbs4.com/local/hyperbaric.martinizi.francesco.2.1040957.html].
4. The authors state that HBOT has a more favorable cost-to-benefit ratio than speech, occupational and behavioral therapies: we strongly disagree with such a statement. Although imperfect, speech/language and occupational therapies, behavioral strategies based on applied behavior analysis, and cognitive behavioral therapies have substantial and replicated scientific evidence and sustainable outcome data to support their use, whereas HBOT does not. The authors’ present study was not designed to compare HBOT to such therapies, and therefore, such a comparison should not be contained in the conclusions of the authors’ study or response. Speech/language, occupational and behavioral therapies are often supported by school district/educational funding, whereas HBOT is not. To imply that HBOT can substitute for speech/language, occupational or behavioral therapies is irresponsible of the authors.
5. There is another posting questioning the “reasonable” costs of HBOT. Can the authors provide a range of costs per treatment by geographic region, the usual numbers of treatments that are provided over the course of a year, and the length of therapies their centers are advising?
6. There is a difference between statistical significance, biological significance and clinical relevance, particularly using a nonspecific and subjective primary endpoint of a clinical global impression. Additionally, there were no true objective or observational outcome measures, and there were other subjective outcome measures that did not change significantly, some using scales that are yet to be validated. Although an interesting study, our opinions remain that this clinical trial does not support the use of HBOT for children with autism.
Mark Mintz, M.D. The Center for Neurological and Neurodevelopmental Health and the Clinical Research Center of New Jersey
Jonathan W. Mink, M.D., Ph.D University of Rochester School of Medicine and Dentistry
Max Wiznitzer, M.D. Rainbow Babies and Children’s Hospital
Competing interests
None declared
The Authors Respond: Results of the Study are Positive
Daniel Rossignol, International Child Development Resource Center
26 October 2009
We again thank Drs. Mintz, Mink, and Wiznitzer for their comments on our article reporting our double-blind controlled study demonstrating the effectiveness of hyperbaric treatment in children with autism. We address their additional concerns as follows: 1. Since the CGI-I is a scale of improvement, the only scores obtained for the physician CGI-I scale were generated at the end of the study. To obtain these scores, the physician rated the child’s performance at the end of the study compared to the physician’s baseline exam of the child. This evaluation generated a score for the change in overall functioning compared to baseline for each child. The statistical analysis then compared the scores in the treatment group to the scores in the control group, and therefore there was no intragroup comparison performed, but rather an intergroup comparison. The description of this analysis is found on page 7 of the study.
2. In reviewing these concerns about the p-value calculation for the CGI-I physician scale, we realized that an error appears on page 6 of the paper. In the “Analysis” section, we initially wrote “All data … are presented as mean ± SEM (standard error of the mean)” when we meant to write that “All data … are presented as mean ± standard deviation.” Therefore, for the physician CGI scale, a mean score of 2.87 with a standard deviation of 0.78 was found in the treatment group (30 children) compared to a mean score of 3.62 with a standard deviation of 0.75 in the control group (26 children). Using this statistical data, an independent t-test demonstrates t = 3.65, with a degree of freedom of 54, and corresponds to a two-sided p-value of 0.0006 for the physician CGI-I. This same p-value was calculated from the raw data, and this raw data was also recently sent to an independent and blinded statistician who confirmed this p-value. The p-value for the ATEC sensory/cognitive awareness subscale (p = 0.0367) was also confirmed by an independent and blinded statistician using the Wilcoxon Rank Sum test. All other scales in the paper originally used standard deviations and thus their associated p-values remain correct as originally reported. We thank Drs. Mintz, Mink, and Wiznitzer for drawing our attention to this error, and regret any confusion it has caused.
3. In their statements, Drs. Mintz, Mink, and Wiznitzer state their “concerns about harm were not specifically directed at HBOT” but then go on to cite the tragic death of a child and grandmother using HBOT via a vastly different protocol, pressure, and oxygen level. They are also concerned about families “pursuing treatments that have not undergone more rigorous scientific scrutiny.” However, since the only FDA-approved treatment at this time for autism is risperidone (for the treatment of irritability), almost every available treatment for autism is arguably in need of further study, including hyperbaric treatment.
4. In neither our paper nor our response did we state or imply that hyperbaric treatment could replace speech, occupational, or behavioral therapy. Nor did we state that hyperbaric treatment has a better cost-to-benefit ratio. Instead, we have encouraged exploration and consideration of the costs, benefits, and risks of hyperbaric treatment and, given that all of the authors involved in this trial routinely recommend speech/language, occupational, educational, and behavioral therapies for children with autism, we routinely encourage consideration of the costs and benefits of these therapies as well.
5. In our previous response, for hyperbaric treatment performed using the treatment parameters in this study, we provided a cost estimate of $40-60 per 60 minute treatment (varying based upon geographic region). Although the total number of treatments usually provided varies based upon physician recommendation, severity of illness, and several other factors, most children who initiate hyperbaric treatment using the treatment parameters in this study complete 40 hyperbaric treatments. Further recommendations for treatment are often based upon the child’s performance before, during, and shortly after that initial treatment period.
6. Finally, the primary outcomes used in our study are ones often employed in drug studies in children with autism. For example the primary outcome of a recent study of citalopram was the Clinical Global Impression—Improvement scale [1]. The other two scales used in our study have also been employed in other studies in autism. Therefore, we stand by our original conclusion that hyperbaric treatment for some children with autism can be beneficial.
Dan Rossignol Lanier Rossignol Scott Smith Cindy Schneider Anju Usman Jim Neubrander Eric Madren Gregg Hintz Barry Grushkin Elizabeth Mumper
Reference
1. King BH, Hollander E, Sikich L, McCracken JT, Scahill L, Bregman JD, Donnelly CL, Anagnostou E, Dukes K, Sullivan L et al: Lack of efficacy of citalopram in children with autism spectrum disorders and high levels of repetitive behavior: citalopram ineffective in children with autism. Arch Gen Psychiatry 2009, 66(6):583-590.
Competing interests
Authors of original article
ppO2 seems to be the key factor
Herb Martin, LearnQuick.Com
13 April 2010
Presuming for the sake of discussion that treatment is effective.
As the grandparent of an autistic child and a former commercial diver I have great personal interest in this subject, and knowledge of diving gases and physiological effects.
After a serious Internet search, I can find no evidence, nor any serious claim, of any proposed mechanism for these effects EXCEPT for the increased partial pressure of O2 (ppO2).
Given that an oxygen concentrator can produce 95% O2 (perhaps as low as 80% actually delivered) and that the treatment dives of this study are only to 1.3 ATA -- or in home devices up to 1.6 ATA -- then we need to seriously consider the idea of treatment with O2 at standard pressure.
1.3 - 1.6 with only slightly increased O2 percentages (e.g, 24%) is MUCH lower than treatment with 80-95% O2 outside of a chamber.
Now clearly using 95% O2 AND 1.6 ATA chamber pressure MIGHT provide additional benefits over O2 alone, but for the cost, time (two adults, a chamber operator AND a companion for the child), slightly increased risks, lowered convenience, etc, we need to know if O2 alone is sufficient/effective/equivalent.
80-95% O2 will provide 4-5 times the normal ppO2, while the study protocols were doing well to reach 1.5 times.
Home or small clinic chambers with O2 might provide about 7 times the ppO2, but are the disadvantages worth the extra cost and trouble?
We have an effective upper limit of 100% O2 and 2 ATA pressure, giving 2.0 ppO2 or roughly a 10 times increase over ambient air.
This is due to both the design of small treatment chambers and the significant increase in central nervous system O2 toxicity beyond 2 ATA ppO2.
Is 0.90 ATA ppO2 effective? More effective than .3 ATA or so?
Is 1.4 pp02 necessary? What is the minimum that gives most of the benefits? How close to 2.0 is most effective?
A $2000 (or less) O2 concentrator is MUCH EASIER to afford and deal with than a $15,000-20,000 chamber (including a concentrator) and much more portable.
Also, this would make it even easier for clinics to offer treatment -- one tech could likely monitor several pairs of child-parent patients, instead of requiring full commitment of attention and a chamber to just one set.
[Of course a much larger chamber could allow for multiple patient sets to dive, but then there are other issues involved.]
We all want to HELP these children -- these costs are real, but trivial if the benefits are real.
Competing interests
My grandson is autistic and a patient of Dr. Rossignol, my grandson anecdotally seems to benefit from the compression treatments -- I have made 5 dives with him myself, and his parents spent significant money on both in clinic treatments and on a home chamber.
Questions for the authors
1 April 2009
Was the treatment pressure for the control group 1.3 atmospheres gauge pressure or 1.3 atmospheres absolute? From the wording of the article it appears as if it was 1.3 atmospheres absolute, or a depth equivalent of 10 feet of sea water. If this is the case, then the treatment group received an O2 partial pressure of about 0.31 atmospheres absolute, or a surface equivalent of 31% O2. If the treatment pressure was 1.3 atmoshperes gauge (2.3 atmospheres absolute, or a depth equivalent of 43 feet of sea water), then the treatment group received an O2 partial pressure of 0.55 atmospheres absolute, or a surface equivalent of 55% oxygen. Both of these can be delivered to a patient without the use of a hyperbaric chamber. Was any consideration given to using a second control group which received surface oxygen at a partial pressure equivalent to that delivered to the hyperbaric treatment group?
Competing interests
None declared
Hyperbaric Oxygen
1 April 2009
24% Oxygen at 1.3 atmospheres is equivalent to 31.2% at 1 atmosphere. This is not by any normal usage hyperbaric oxygen. The same Oxygen levels could have been achieved at less distress to the children and less cost with a simple face mask or even in a small room. If the pressure as opposed to the oxygen is the dimension of interest then 1.3 atmospheres at 21% could have been compared with 27.3% at ambient pressure. Autism is a terrible affliction and we owe the sufferers our best efforts.
Competing interests
A commitment to offering ill people only interventions with some plausible likelihood of benefit.
Two Small Points
1 April 2009
This article is quite interesting. While on the surface it seems to lend some scientfic validation for a possible treatment of autism, it may raise questions - about validity of the results based on the methodology employed, and possible clinical and economic implications of the therapy described.
Firstly, the stated treatment pressure (1.3ATM) would seem highly unlikely to be uniformly achievable across all study locations with the equipment that appears to have been used.
As described in the section titled, "Interventions":
"These procedures included covering control switches, inflating and deflating the chambers to simulate pressure changes, and masking the sounds from the chambers."
The use of "inflatable" monoplace chambers, sometimes known as "portable" hyperbaric chambers, is a clear indication that the actual total pressures (and results of this study) would have been affected by the ambient air pressures at the times and locations of treatment.
It appears that the study sites vary from low lying cities such as Edison, NJ (115'AMSL) and Melbourne, FL(33'AMSL) to cities of modest elevation such as Lynchburg, VA (938'AMSL) and Phoenix, AZ (1135'AMSL).
Even if it is assumed that the chambers at the different sites operate with similar precision, and it is also assumed that they fully pressurize to 4.4 PSI (.3 ATM), the actual gauge pressure that would be required to deliver the full stated 1.3 ATM at sea-level for an inflatable chamber (1 ATM of ambient pressure + .3 ATM added pressure = 1.3 ATM, or more accurately, 1.3 ATA), there could be a 10% or more deviation in added treatment pressure between several sites due to elevation alone.
Local atmospheric pressure is typically reported as sea-level pressure1 for aviation, and weather maps, etc., but the actual station pressure is affected by the elevation. As an example, calculated station pressure (the ambient air pressure) at Melbourne, Florida would be approximately 14.68 PSI, and with 4.4 PSI of added treatment pressure (.3 ATM), the actual total pressure would be the stated 1.3 ATM (more accurately 1.3 ATA). The same treatment in Phoenix, Arizona would be quite different. Calculated station pressure (the ambient air pressure) there, would be approximately 14.13 PSI, and with 4.4 PSI of added treatment pressure (.3 ATM), the actual total pressure would be only 1.26 ATA.
With a theoretical maximum pressure increase of 4.4 PSI (.3 ATM) above the ambient pressure, a potential error of .04 ATM (by assuming ambient sea-level pressure exists at elevation when it probably does not) would mean that 13% of the expected pressure increase above 1 ATM was never delivered. This may not be insignificant at such low added pressures in terms of total oxygen delivery. If the added treatment pressure of the inflatable chambers is less than the full 4.4 PSI, the error grows. Furthermore, the same problems with calculating actual treatment pressures will apply to the "placebo" control group.
Secondly, the authors of this article should be commended for fully disclosing their competing interests (derivation of revenue from hyperbaric treatment in their practices), in light of what the results of this study may mean if replicated.
The oxygen delivered at 24% FIO2 and 1.3 ATA via a portable hyperbaric chamber can be easily matched and even surpassed with simple oxygen therapy and no hyperbaric chamber at all (e.g. a simple face mask delivering 35-55% FIO2 at 5-8 lpm2 @ 1 ATA). While the authors cite an in vitro study for a hypothetical effect of pressure alone, the notion of in vivo and clinical significance in autism from such a small pressure change in and of itself doesn't seem to have much (if any) scientific support. All things considered, the results of this study suggest that simple oxygen therapy (at a fraction of the cost of hyperbaric oxygen therapy with inflatable chambers), may be an effective autism treatment.
1http://www.nws.noaa.gov/oso/oso1/oso12/fmh1/fmh1ch11.htm
2National Institutes of Health - Critical Care Medicine Department, Critical Care Therapy and Respiratory Care Section Category: Clinical, Section: Medicinal Gas Therapy, Title: Oxygen Therapy Procedure, Policy: 01, Revised: 02/02
Competing interests
Financial Competing Interests: None
Personal Compteting Interests: I'm the father of an autistic child, and author of a non-commercial blog that often examines autism treatment claims with scientific skepticism.
Normobaric oxygen vs Hyperbaric Oxygen
6 April 2009
Oxygen dosing (which is modulated by pressure far more than absolute concentration)is what provides the signaling at a cellular level, including the subcellular level in the mitochondria, to facilitate the changes observed using a hyperbaric environment.
No where was this made more clear than in a 163 patient prospective study of carbon monoxide poisoning patients who were given different doses of supplemental normobaric oxygen. Six week post-treatment cognitive sequelae were unchanged - contrast that to the results when hyperbaric oxygen is used. (Weaver et al in reply to Scheinkestal et al: The role of hyperbaric oxygen in carbon monoxide poisoning. Emerg Med Australa 2004;16;394-399).
The gas laws and hyperbaric medicine are not taught in medical school, so it is unfortunate that these questions have to come up, because there is a fundamental lack of understanding about what oxygen under pressure does. This problem is far greater than treating autistic children - this endemic lack of understanding about what hyperbaric oxygen does and when to use it has kept hyperbaric chambers out of Emergency Departments everywhere when there should be a chamber in every trauma center.
In the case of autistic children and children with CP, the therapeutic window, or that pressure that will effect DNA signaling, appears to be very low - in many cases less than 1.3 ATA. That has been my experience with dozens of children so affected. However that is not the appropriate pressure (oxygen dose) for treating other conditions.
Oxygen dose is poorly understood in hyperbaric medicine and even less understood outside of the field, again there is a lot of misapplication. Breathing in normobaric oxygen, regardless of the concentration is not comparable to using oxygen under pressure.
Competing interests
President of the International Hyperbaric Medical Association
Not just Oxygen?
6 April 2009
Hopefully this study will be repeated in a multi-center trial in academic medical centers as it does seem promising, with some added neurofunctional imaging such as SPECT, PET or fMRI with MRS. The difficulties of shamming or placebo grouping this intervention seems to have plagued the idea since its first inceptions for medical therapy. I simply wonder on the basis of having read the following article whether too much focus has been placed on oxygen as the sole therapeutic effector in this treatment modality?
Stem cell mobilization by hyperbaric oxygen.
Thom SR, Bhopale VM, Velazquez OC, Goldstein LJ, Thom LH, Buerk DG.
Am J Physiol Heart Circ Physiol. 2006 Apr;290(4):H1378-86. Epub 2005 Nov 18.
PMID: 16299259 [PubMed - indexed for MEDLINE]
Competing interests
Child with an autistic disorder and research/academic interests in HBOT.
hyperbaric vs. normobaric oxygen (response)
16 April 2009
Dr. Stoller's prolific writing and his efforts toward educating the public on hyperbaric medicine are respectfully acknowledged. I would disagree with his implication that the mechanism of action of hyperbaric oxygen is poorly understood in the hyperbaric community. Granted, some of the biomechanics of oxygen at the cellular and subcellular level remain unclear. However, Dalton's Law, Henry's Law and the alveolar gas equation are familiar to most of us, and they all support the contention that 0.31 ATA of oxygen is 0.31 ATA of oxygen no matter what the delivery method. Perhaps there is an as-yet-unknown synergistic effect of slight increases of ambient pressure on "normobaric" oxygen pressures, i.e. partial pressures of O2 that are less than or equal to 1.0 ATA (as this study suggests). If so, this effect needs to be carefully studied and documented, not merely taken for granted. Legitimate scientific questions will arise from the medical community at large, and we must be prepared to answer them if we are to be perceived as credible. We need to control for all variables, not just those that appear to prove our postulations.
Competing interests
None declared
Gas Laws Not Mysterious - They Apply Here
16 April 2009
Comments by Mr. Stoller indicate that oxygen dosing is more modulated by pressure than absolute concentration. While that may be especially true when concentrations of O2 (FIO2) approach 100%, the statement does not apply with respect to this study. In fact, inflatable chambers have built-in pressure relief valves that, unless modified (illegally), limit the total pressure at about 4psi above ambient atmospheric pressure. Without the ability to add additional pressure above this very low maximum, the ability to "modulate" pressure, except towards even less pressure (and lower oxygen dose), does not even exist. Citation of Weaver et al. fails to add relevance with respect to this study (or autism) - The Weaver et al. treatment group seems to have received real hyperbaric oxygen therapy (100% 02 at 3 and 2 ATA).
In response to Mr. Stoller's comments regarding hyperbaric medicine in general, the gas laws are taught in U.S. medical schools, in the sections on pulmonary physiology and again during blood physiology. Hyperbaric oxygen therapy is also discussed during these sections.
Mr. Stoller writes, "Breathing in normobaric oxygen, regardless of the concentration is not comparable to using oxygen under pressure."
When dealing with added pressures that are so low (like the .23-.27 ATM in this study), and FIO2 of 24%, they are quite probably very comparable.
Unless there is a known and proven clinical effect of such low pressure alone for autism (which doesn't appear likely at this point), the only factor that provides hypothetical effect is the increase in partial pressure of oxygen. Cellular physiology is dependent on oxygen partial pressure, which is the product of the total pressure and the fraction (percentage) of oxygen inspired. Total pressure - apart from its impact on partial pressure - has negligible effect on oxygen delivery.
One additional question for the original authors: Can you provide more detail about how the multiple significance tests might have corrected for?
Competing interests
Financial Competing Interests: None
Personal Competing Interests: I'm the father of an autistic child, and author of a non-commercial blog that often examines autism treatment claims with scientific skepticism.
correction
17 April 2009
In my first comment, I mistakenly asked if the treatment pressure for the CONTROL group was 1.3 atmospheres absolute or 1.3 atmospheres gauge. My question was about the TREATMENT group. I regret any misunderstanding that may have arisen as a result.
Competing interests
None declared
Results of the Study are NOT Positive as Claimed
7 May 2009
Dear Editor/Authors:
We read with interest the recently published article by Rossignol et. al.(1) concerning the use of hyperbaric oxygen treatment (HBOT) in children with Pervasive Developmental Disorder-Autistic Disorder. As the authors have rightly pointed out, there is a need for more evidenced-based treatments for this distressing disorder, and they should be commended for undertaking a controlled clinical trial. But we are concerned that the evidence as presented by these authors does not provide support for the use of HBOT in this population despite the authors’ conclusions and claims to the contrary. In particular, we believe there are significant methodological and statistical flaws in the design, implementation and interpretation of the results of this clinical trial, which have rendered the conclusions of the authors faulty and mistaken. In particular, we invite the authors to respond to the following:
1. In the Methods section, it is stated “…At the end of the study, all children assigned to the control group were offered 40 hyperbaric treatments at the treatment pressure (1.3 atm and 24% oxygen) if the parents desired (all parents chose this option; treatments were provided without charge)...” Essentially, this is a break of the blind, as the investigators and families would know who were in the control group prior to the completion and analysis of the study for all participants. If an open label extension was to be offered, all participants enrolled in the double blind treatment phase would need to be offered the open label treatment to maintain the integrity of the double blind.
2. It is legitimate to provide an analysis of the intent-to-treat (ITT) group as was done in this paper, but this should be done in conjunction with an analysis of the per-protocol (PP) group to provide valid analyses and conclusions. This is particularly necessary in this study, as a primary endpoint measure of the study, the Autism Treatment Evaluation Checklist (ATEC), could not be done for approximately 30% of the total study group as a result of a protocol violation, as was stated in the Methods “…Due to an administration error, the baseline ATEC was not performed at one of the study centers, and thus data was available for analysis for 23 children in the treatment group and 21 children in the control group...” Additionally, it is concerning that there was one site of six that accounted for 30% of recruitment, and yet could not properly perform the protocol procedures for collecting primary endpoint data. What was the integrity of the remainder of the data from that site? Why was there not a better distribution of enrollment across sites?
Thus, of the N = 62 subjects enrolled/randomized, only N = 44 had an ATEC done, with an additional seven dropouts, resulting in a PP group of N = 37. What were the results of the data for the PP group? How does the PP group compare to the ITT group? Thus, with a PP group of N = 37, it is likely that the PP group is not powered sufficiently, as the authors have stated a sample size of N = 43 would be necessary for an effect size of d = 0.44.
3. We have concerns over the statistical methods and presentation of the data. In Figures 2-4, there are no error bars, which if used would graphically display the lack of significant differences between the treatment and control groups. Figure 2 is very confusing: how can there be a representation of a change from baseline for the CGI-I, when it is a scale that is given only at the end of the treatment phase, as stated in the Methods section, and thus, no baseline for comparison? The data as presented indicate only a slight intergroup difference in the subjective Clinical Global Impressions-Improvement (CGI-I) as completed by the physicians, but not the families; one subscale of the ABC; and one subscale of the ATEC. Yet, there is a wide error margin for the scoring, and there is no Effect Size calculation, and given the small sample size, the Effect Size is likely insignificant.
4. In the listing for this clinical trial on www.clinicaltrials.gov, it is stated that there is a Blinded Physician for the CGI (no mention of a parent CGI), Blinded Therapist for the ABC, and Blinded Therapist for the ADOS. The ATEC is listed as being given by the parent, but it is not noted that the ATEC evaluator is blinded, and no mention of a physician or therapist ATEC rater. Yet, in the Methods section, it is stated that the ABC, CGI and ATEC were done by the parent or caretaker, the CGI by the physician, and no mention of any Therapists administering any of the scales. Why this contradiction? Were the scales done as a structured interview? Was there any prior training or inter-rater reliability analyses of the scales prior to site initiation to improve the validity and consistency of results? Why was “placebo” taken out from “placebo control group” in 2007?
5. In the opening Abstract, the authors conclude that there were significant improvements in “…overall functioning, receptive language, social interaction, eye contact, and sensory/cognitive awareness...” This is misleading, as the authors are referring to an intragroup analysis of the treatment group only. This is irrelevant to the design and power calculations of the study, which was designed to compare a treatment group to a control group. When analyzing the results of the comparative analyses, there is only a slight difference noted as described in paragraph #3 (above), with the additional confounder that the ATEC could be analyzed for only 60% of the ITT group.
6. In looking at the peer reviews of the paper, there were concerns about table 2 in the original submission, with Reviewer #2 having trepidation that there were too many variables being analyzed, which can lead to false statistical conclusions via the Bonferroni effect (the chance of finding a positive result is linearly proportional to the number of statistical correlations).(2) Why did the authors eliminate this table of data from the final publication, rather than allowing the reader to see all the data?
7. There is an extensive Discussion section that purports to provide a physiological basis for the allegedly “positive” results of this clinical trial, namely a reversal of “cerebral hypoperfusion” or suppression of “inflammatory” mechanisms. However, this clinical trial did not include any surrogate markers for assessing cerebral perfusion or immunological status, and thus, such an extensive discussion is more applicable to a review paper than a paper reporting results of a clinical trial with inconclusive results.
8. In their conclusions, the authors state that HBOT is “…feasible and not necessarily costly…” but have not provided any foundation for such a statement. It is the authors’ experience that patients who have tried HBOT have spent large sums of money not reimbursable by insurance. What is the average cost of 40 treatments over four weeks across the six participating centers?
The authors rightly conclude that it is necessary to await the results of other ongoing HBOT studies from other centers before there can be any statements that HBOT is efficacious or safe for children with Autism Spectrums Disorders. Thus, it is perplexing that the authors subsequently conclude “…in light of the positive results of this study…hyperbaric treatment appears to be a promising treatment for children with autism…” The results from this study do not support such a statement.
A critical reading and analysis of the data as presented by Rossignol et. al. has provided evidence that HBOT does not benefit children with Autism or Autism Spectrum Disorders, and at this point in time, HBOT is not a justified or evidence-based therapeutic intervention. Yet, the authors provide antithetical and misleading conclusions. Furthermore, this clinical trial has major methodological and statistical flaws and protocol violations that render the few questionable but clinically insignificant positive statistical correlations meaningless and inconsequential. Families of children with Autism Spectrum Disorders must deal with difficult decisions for weighing a large number of intensive treatment options that subject their loved ones to potential harm, as well as emotional and financial stress. It is important for the medical community to continue to explore new and promising therapies for this population. However, this study, despite its misleading conclusions, does not provide sufficient support or evidence for the use of HBOT for children with Autism Spectrum Disorders.
We await the authors’ response.
Mark Mintz, M.D.
The Center for Neurological and Neurodevelopmental Health
and the Clinical Research Center of New Jersey
Jonathan W. Mink, M.D., Ph.D
University of Rochester School of Medicine and Dentistry
Max Wiznitzer, M.D.
Rainbow Babies and Children’s Hospital
1. Rossignol DA, Rossignol LW, Smith S, et al: Hyperbaric treatment for children with autism: a multicenter, randomized, double-blind, controlled trial. BMC Pediatrics 2009, 9:21; doi:10.1186/1471-2431-9-21
2. Shaffer, J. P. "Multiple Hypothesis Testing." Ann. Rev. Psych. 46, 561-584, 1995.
Competing interests
No Financial Competing Interests
The Authors Respond: Results of the Study are Positive
9 June 2009
We thank Drs. Mintz, Mink, and Wiznitzer for their comments on our article reporting our double-blind controlled study demonstrating the effectiveness of hyperbaric treatment in children with autism. We must point out fundamental errors in their review of our study, its methodology, and its data analyses, however, and clarify any misinterpretations of our findings.
1. With regard to the blinding procedure, only after all study data had been collected, taken to a central location, and analyzed were the control group patients unblinded and offered 40 hyperbaric treatments. These subsequent additional treatments were provided to the controls at a location remote from all study sites. Thus, the integrity of the double-blind procedure was kept completely intact throughout both the data collection and analysis stages. Consequently, there was not “a break of the blind” as the commentators contend.
2. An administrative error occurred at one site, which did not perform the ATEC at the beginning of the study. No other protocol violations occurred at that site or any other sites. Thus, the ABC, CGI scales, and all other study protocols were performed properly at all sites. Despite obtaining ATEC data for only 44 of the 56 children due to the administrative error, there was a significant change on the ATEC sensory/cognitive awareness subscale (p = 0.0367; effect size = 0.55) in the treatment group compared to the control group. The administration error did not affect the CGI or ABC scales, and data for these scales was collected on all children in the study. The site with this administration error enrolled 12 children in the study (19% of the children in the study; the distribution of enrollment across sites was fairly even). None of the 44 children who received an ATEC dropped out of the study and therefore the commentators have wrongly subtracted 7 dropouts from the 44, and thus the number of 37 is erroneous. Since 56 children were analyzed in the study, the ATEC was performed on 44/56 children (79%), who were analyzed in accordance with Figure 1. Because all 44 children were properly accounted for, the “per-protocol” analysis for the ATEC is the same as the intention-to-treat analysis reported in the article. Thus the per-protocol analysis was of sufficient power and effect size.
3. Prior to initiating hyperbaric treatment, each child was examined by a physician who observed and noted the child’s performance/behavior on the subscales of the CGI at the baseline exam. At the conclusion of the 40 treatments, the physicians repeated their assessments, completing their clinical global impression as compared to baseline. The resulting scores were then used for the statistical analysis (found on pages 7 and 8) in which the mean scores on the CGI between the treatment group and the control group were compared. Figure 2 visually represents the change in score from baseline and does not present any new data. The p-values listed in Figure 2 are the same as those found on pages 7 and 8 and were derived from an intergroup comparison (treatment group compared to control group). Contrary to the comments of Drs. Mintz, Mink, and Wiznitzer, these improvements were statistically significant. Please see the discussion of point #5 below for further comments.
There are no error bars on the figures, but there is no data reported in the figures that can not be obtained from the results section or the tables. Furthermore, all of the scores in the paper are reported with standard deviations, as found on pages 7-9 and the tables. The effect sizes were not reported in the article, but are listed here. The results are actually robust: for the physician CGI score for overall functioning (effect size = 1.0); for the parental CGI score for overall functioning (effect size = 0.62); and for the ATEC sensory/cognitive awareness subscale (effect size = 0.55).
4. The clinicaltrials.gov website is not intended to include intricate details of a study, but instead serves as a registry of studies. Registration of a study on that site is required by most journals for publication of a corresponding article. For this study, a blinded psychologist completed a pre- and post-ADOS scale and observed no significant change in autism diagnosis between the two groups (this data is found on page 13 of the article; the ADOS was also used to confirm the diagnosis of autistic disorder prior to enrollment in the study). The ATEC and ABC are parent-generated data gathering tools (i.e., see reference [1]), and parents were given instructions regarding their proper use. All of the evaluators for all of the scales in this study (parents, physicians, and psychologists) were blinded throughout the entire study. The term “placebo” was originally listed in the clinicaltrials.gov website. Most hyperbaric studies that are controlled have used 1.1 atm as a “placebo” (for example, see reference [2]). Since we provided some pressure in the control group, we came to the conclusion that this was not a true placebo but a “near-placebo” as described in the abstract and on page 2. However, given the example provided by other hyperbaric studies, we would have been justified to call the control pressure a “placebo” if we had desired.
5. It appears that comment #5 lies at the heart of the commentators’ assertion that this study was “NOT Positive as Claimed.” Careful readers will note a number of significant improvements in the treatment group versus the control group, and that all of the analyses for the improvements listed in the conclusion of the abstract (“overall functioning, receptive language, social interaction, eye contact, and sensory/cognitive awareness”) were performed using an intergroup analysis (comparing the treatment group to the control group) not “an intragroup analysis of the treatment group only” as the commentators state.
The abstract clearly and plainly states “After 40 sessions, mean physician CGI scores significantly improved in the treatment group compared to controls in overall functioning (p = 0.0008), receptive language (p < 0.0001), social interaction (p = 0.0473), and eye contact (p = 0.0102); 9/30 children (30%) in the treatment group were rated as "very much improved" or "much improved" compared to 2/26 (8%) of controls (p = 0.0471); 24/30 (80%) in the treatment group improved compared to 10/26 (38%) of controls (p = 0.0024).” This is clearly an intergroup analysis (which is also described on pages 7 and 8 of the study), and its results are statistically significant. There was also significant changes on the CGI as rated by parents when comparing scores of the treatment group to the control group, as described on pages 7 and 8 and as found in the abstract: “Mean parental CGI scores significantly improved in the treatment group compared to controls in overall functioning (p = 0.0336), receptive language (p = 0.0168), and eye contact (p = 0.0322).” Therefore, there were statistically significant improvements using an intergroup analysis, in opposition to what the commentators stated in this point. On the ABC, the abstract also gives results for the intergroup analysis: “In the treatment group compared to the control group, mean changes on the ABC total score and subscales were similar except a greater number of children improved in irritability (p = 0.0311).” Finally the change on the ATEC sensory/cognitive awareness subscale was significant and was performed using an intergroup analysis: “On the ATEC, sensory/cognitive awareness significantly improved (p = 0.0367) in the treatment group compared to the control group.” As described above, the ATEC was performed on 79% of the 56 children who were eligible for analysis, not 60% as the commentators state. Most notably, rather than a “slight difference” as alleged by the commentators, we measured statistically significant (intergroup) improvements in the treatment group when compared to the control group in these measures. Thus, the only reasonable conclusion is that the studied treatment was positive.
6. The scales we used for comparisons between groups in this study were based upon our two previous studies using hyperbaric treatment in children with autism. Thus, we used planned comparisons in the current study, making the statistical analysis stronger because it was hypothesis based. The Bonferroni correction is one method for correcting for inflation of the alpha when conducting multiple statistical tests. However, when the dependent measures are correlated, as commonly occurs in clinical trials, the Bonferroni correction becomes too conservative and is not appropriate. Since, in such cases, there is little agreement on the precise method for calculating a statistical correction, some investigators suggest using a global assessment measure rather than using a Bonferroni correction [3]. This is exactly what we did in the current study. Indeed, we used the CGI score to confirm change in overall functioning as compared to baseline. Furthermore, during the review process, Reviewer #2 indicated that he did not “feel adequately qualified to assess the statistics” and suggested a “review by a statistical expert.” Prior to publication, the journal obtained an independent statistical review of the study’s data, which confirmed our findings.
7. The discussion section is meant to discuss the implications of the results of a study and possible reasons for the outcome, and so the discussion in this article is traditional, expected, and entirely reasonable. As requested by reviewers, we added further material to the discussion section concerning the possible “physiological basis” for the improvements found in this study. As noted in our discussion, we could not be certain what effect the hyperbaric treatment had on markers of inflammation or hypoperfusion, since we did not measure markers of these particular parameters in this study.
8. The average cost for hyperbaric treatments using the parameters performed in this study varies depending on the location in the country, but a reasonable estimate is approximately $40-60 per 60 minute treatment. The commentators, some of whom in actuality do have competing financial interests in therapy-based services performed at their respective centers, make a valid point to discuss cost and benefit analysis. While we recognize the importance of speech, occupational, and applied behavioral analysis therapy, we contend that hyperbaric treatment may actually be less costly with respect to the benefits obtained. Of course, the cost of hyperbaric treatment should be considered in the risk, benefit, and cost analysis discussion between the family and clinician, and decisions made based on the circumstances of each individual patient.
We did not state or conclude that “it is necessary to await the results of other ongoing HBOT studies from other centers before there can be any statements that HBOT is efficacious or safe for children with Autism Spectrum Disorders.” The safety of this treatment (using the parameters in this study) is established given the results of this study and previously published studies in autism [4-6]. Furthermore, when performed properly, this treatment has been studied in children and proven to be safe at higher pressures and oxygen concentrations than those used in this trial [7]. Since this is a double-blind controlled trial with statistically significant results, the efficacy of hyperbaric treatment in autism is now more proven than not. On page 13, we did state that “further studies are needed by other investigators to confirm these findings” but that statement does not mean we think that hyperbaric treatment (at the parameters used in this study) is not safe or not efficacious. We realize that other studies will need to replicate these findings before most physicians will change their practice and treatment recommendations. Based upon our findings, and the previously published studies of hyperbaric treatment in children with autism, it is completely reasonable to conclude that this treatment is “promising.”
A close review of the comments by Drs. Mintz, Mink, and Wiznitzer demonstrates significant errors in their interpretation. To reiterate, the analyses finding significant improvements in this study (i.e., p < 0.05) clearly were performed using an intergroup analysis (treatment group compared to control group), not an intragroup analysis as the commentators allege.
Finally, we are very aware of the difficult decisions that families of children with autism need to make concerning possible treatment options, and, in fact, designed and performed this study to help provide further objective information upon which families could base such difficult decisions. We are confused as to why the commentators state, without support, that the treatment we have evaluated would demonstrate “harm” to the patient, as we are not aware of any evidence of harm that has come to a child using hyperbaric treatment within the parameters used in this study. In fact, this study and the previous published studies of hyperbaric treatment in children with autism (using the protocol in this study) have reported that this treatment is safe.
In conclusion, we maintain our position that hyperbaric treatment using the parameters in this study is a cost-effective and promising medical adjunctive intervention to ameliorate autism symptomatology.
Dan Rossignol
Lanier Rossignol
Scott Smith
Cindy Schneider
Anju Usman
Jim Neubrander
Eric Madren
Gregg Hintz
Barry Grushkin
Elizabeth Mumper
References
1. McCracken JT, McGough J, Shah B, Cronin P, Hong D, Aman MG, Arnold LE, Lindsay R, Nash P, Hollway J et al: Risperidone in children with autism and serious behavioral problems. N Engl J Med 2002, 347(5):314-321.
2. Clarke RE, Tenorio LM, Hussey JR, Toklu AS, Cone DL, Hinojosa JG, Desai SP, Dominguez Parra L, Rodrigues SD, Long RJ et al: Hyperbaric oxygen treatment of chronic refractory radiation proctitis: a randomized and controlled double-blind crossover trial with long-term follow-up. Int J Radiat Oncol Biol Phys 2008, 72(1):134-143.
3. Feise RJ: Do multiple outcome measures require p-value adjustment? BMC Med Res Methodol 2002, 2:8.
4. Rossignol DA, Rossignol LW: Hyperbaric oxygen therapy may improve symptoms in autistic children. Med Hypotheses 2006, 67(2):216-228.
5. Rossignol DA, Rossignol LW, James SJ, Melnyk S, Mumper E: The effects of hyperbaric oxygen therapy on oxidative stress, inflammation, and symptoms in children with autism: an open-label pilot study. BMC Pediatr 2007, 7(1):36.
6. Chungpaibulpatana J, Sumpatanarax T, Thadakul N, Chantharatreerat C, Konkaew M, Aroonlimsawas M: Hyperbaric oxygen therapy in Thai autistic children. J Med Assoc Thai 2008, 91(8):1232-1238.
7. Ashamalla HL, Thom SR, Goldwein JW: Hyperbaric oxygen therapy for the treatment of radiation-induced sequelae in children. The University of Pennsylvania experience. Cancer 1996, 77(11):2407-2412.
Competing interests
Authors of original article
Cost-effective Oxygen Delivery
24 June 2009
Dr. Rossignol writes:
"We are confused as to why the commentators state, without support, that the treatment we have evaluated would demonstrate “harm” to the patient, as we are not aware of any evidence of harm that has come to a child using hyperbaric treatment within the parameters used in this study."
This confusion is understandable, but unfounded. A quick review of the commentators' actual words and context reveals that no such statement, that the treatment evaluated by Rossignol et al. would demonstrate harm, was made.
Dr. Rossignol also notes:
"In conclusion, we maintain our position that hyperbaric treatment using the parameters in this study is a cost-effective and promising medical adjunctive intervention to ameliorate autism symptomatology."
Monthly cost for two $40-$60 daily treatments (60 minutes each, 5 days a week)?
About $2,000
Monthly cost for identical oxygen delivery with simple O2 therapy for same treatment period?
About $200
Competing interests
Financial Competing Interests: None
Personal Competing Interests: I'm the father of an autistic child, and author of a non-commercial blog that often examines autism treatment claims with scientific skepticism.
Response to the Authors’ Response: Results of the study are NOT positive as claimed
12 August 2009
We thank Drs. Rossingol et al. for responding to our posted comment about our concerns with their study’s methodologies, results and conclusions. The authors’ response has not changed our analyses and opinions, and in particular has not changed our conclusion: it is our opinion that the paper does not support the use of hyperbaric oxygen therapy (HBOT) for children with autism. In particular, we invite the authors to explain or comment on the following:
1. In the authors’ response, they have not clarified how a physician-completed Clinical Global Impression-Improvement scale (CGI-I) can be used at baseline and then again after 40 treatment sessions. [In the Methods section, it states that physicians completed the CGI-I after 40 treatment sessions, but in the authors’ response to us there is a claim that there was a baseline assessment as well.] The CGI-I is a scale of improvement, and thus, does not provide a baseline measure, providing only a subjective end-of-study “impression” of change over the course of the protocol. Thus, there cannot be an outcome measure stating there was “no change” at baseline, which is what the authors are using for their statistical comparison [Figure 2].
2. We have tried to recreate the statistical result of the primary outcome measure of the study: the physician-completed Clinical Global Impression-Improvement scale (CGI-I). The data presented under “Outcome Measures” in the text appears to provide for scores represented as a mean +/- standard error of measurement (SEM) for the Treatment Group 2.87 +/- 0.78. Using frequencies for the Control Group, a mean of 3.62 +/- 0.75 is derived. If one uses a two-tailed t-test, then the result is t=0.69 yielding a non-significant p-value of p=0.49. Thus, it is unclear to us how the authors were able to report a p-value of 0.0008. Did the authors use a mean +/- standard deviation, and if so, how was this derived? Again, how can there be a statistical outcome measure compared to “no change” at baseline: how does one surmise there is “no change” prior to any intervention?
3. Our concerns about harm were not specifically directed at HBOT, but more generally at the use of unproven therapies with scant pediatric safety data, causing families to direct limited resources towards unverified therapies, and creating tremendous burdens on family time in pursuing treatments that have not undergone more rigorous scientific scrutiny (many examples exist, but secretin is one recent example: an expensive therapy that became widely used based on scant data, but found to be no better than placebo in a number of subsequent rigorous scientific studies). Possibly HBOT does not have major side effects at the parameters used in this study (and as argued by others on the Comments page, at these parameters, HBOT is likely no better than non-pressurized supplemental oxygen at 23%), but it has sparked false expectations and hope among well-meaning and desperate families in search of a cure for their children, and as such, can contribute to harm in a number of ways, illustrated by the tragic death of an Italian child and grandmother as they sought HBOT in Florida. [http://cbs4.com/local/hyperbaric.martinizi.francesco.2.1040957.html].
4. The authors state that HBOT has a more favorable cost-to-benefit ratio than speech, occupational and behavioral therapies: we strongly disagree with such a statement. Although imperfect, speech/language and occupational therapies, behavioral strategies based on applied behavior analysis, and cognitive behavioral therapies have substantial and replicated scientific evidence and sustainable outcome data to support their use, whereas HBOT does not. The authors’ present study was not designed to compare HBOT to such therapies, and therefore, such a comparison should not be contained in the conclusions of the authors’ study or response. Speech/language, occupational and behavioral therapies are often supported by school district/educational funding, whereas HBOT is not. To imply that HBOT can substitute for speech/language, occupational or behavioral therapies is irresponsible of the authors.
5. There is another posting questioning the “reasonable” costs of HBOT. Can the authors provide a range of costs per treatment by geographic region, the usual numbers of treatments that are provided over the course of a year, and the length of therapies their centers are advising?
6. There is a difference between statistical significance, biological significance and clinical relevance, particularly using a nonspecific and subjective primary endpoint of a clinical global impression. Additionally, there were no true objective or observational outcome measures, and there were other subjective outcome measures that did not change significantly, some using scales that are yet to be validated. Although an interesting study, our opinions remain that this clinical trial does not support the use of HBOT for children with autism.
Mark Mintz, M.D.
The Center for Neurological and Neurodevelopmental Health
and the Clinical Research Center of New Jersey
Jonathan W. Mink, M.D., Ph.D
University of Rochester School of Medicine and Dentistry
Max Wiznitzer, M.D.
Rainbow Babies and Children’s Hospital
Competing interests
None declared
The Authors Respond: Results of the Study are Positive
26 October 2009
We again thank Drs. Mintz, Mink, and Wiznitzer for their comments on our article reporting our double-blind controlled study demonstrating the effectiveness of hyperbaric treatment in children with autism. We address their additional concerns as follows:
1. Since the CGI-I is a scale of improvement, the only scores obtained for the physician CGI-I scale were generated at the end of the study. To obtain these scores, the physician rated the child’s performance at the end of the study compared to the physician’s baseline exam of the child. This evaluation generated a score for the change in overall functioning compared to baseline for each child. The statistical analysis then compared the scores in the treatment group to the scores in the control group, and therefore there was no intragroup comparison performed, but rather an intergroup comparison. The description of this analysis is found on page 7 of the study.
2. In reviewing these concerns about the p-value calculation for the CGI-I physician scale, we realized that an error appears on page 6 of the paper. In the “Analysis” section, we initially wrote “All data … are presented as mean ± SEM (standard error of the mean)” when we meant to write that “All data … are presented as mean ± standard deviation.” Therefore, for the physician CGI scale, a mean score of 2.87 with a standard deviation of 0.78 was found in the treatment group (30 children) compared to a mean score of 3.62 with a standard deviation of 0.75 in the control group (26 children). Using this statistical data, an independent t-test demonstrates t = 3.65, with a degree of freedom of 54, and corresponds to a two-sided p-value of 0.0006 for the physician CGI-I. This same p-value was calculated from the raw data, and this raw data was also recently sent to an independent and blinded statistician who confirmed this p-value. The p-value for the ATEC sensory/cognitive awareness subscale (p = 0.0367) was also confirmed by an independent and blinded statistician using the Wilcoxon Rank Sum test. All other scales in the paper originally used standard deviations and thus their associated p-values remain correct as originally reported. We thank Drs. Mintz, Mink, and Wiznitzer for drawing our attention to this error, and regret any confusion it has caused.
3. In their statements, Drs. Mintz, Mink, and Wiznitzer state their “concerns about harm were not specifically directed at HBOT” but then go on to cite the tragic death of a child and grandmother using HBOT via a vastly different protocol, pressure, and oxygen level. They are also concerned about families “pursuing treatments that have not undergone more rigorous scientific scrutiny.” However, since the only FDA-approved treatment at this time for autism is risperidone (for the treatment of irritability), almost every available treatment for autism is arguably in need of further study, including hyperbaric treatment.
4. In neither our paper nor our response did we state or imply that hyperbaric treatment could replace speech, occupational, or behavioral therapy. Nor did we state that hyperbaric treatment has a better cost-to-benefit ratio. Instead, we have encouraged exploration and consideration of the costs, benefits, and risks of hyperbaric treatment and, given that all of the authors involved in this trial routinely recommend speech/language, occupational, educational, and behavioral therapies for children with autism, we routinely encourage consideration of the costs and benefits of these therapies as well.
5. In our previous response, for hyperbaric treatment performed using the treatment parameters in this study, we provided a cost estimate of $40-60 per 60 minute treatment (varying based upon geographic region). Although the total number of treatments usually provided varies based upon physician recommendation, severity of illness, and several other factors, most children who initiate hyperbaric treatment using the treatment parameters in this study complete 40 hyperbaric treatments. Further recommendations for treatment are often based upon the child’s performance before, during, and shortly after that initial treatment period.
6. Finally, the primary outcomes used in our study are ones often employed in drug studies in children with autism. For example the primary outcome of a recent study of citalopram was the Clinical Global Impression—Improvement scale [1]. The other two scales used in our study have also been employed in other studies in autism. Therefore, we stand by our original conclusion that hyperbaric treatment for some children with autism can be beneficial.
Dan Rossignol
Lanier Rossignol
Scott Smith
Cindy Schneider
Anju Usman
Jim Neubrander
Eric Madren
Gregg Hintz
Barry Grushkin
Elizabeth Mumper
Reference
1. King BH, Hollander E, Sikich L, McCracken JT, Scahill L, Bregman JD, Donnelly CL, Anagnostou E, Dukes K, Sullivan L et al: Lack of efficacy of citalopram in children with autism spectrum disorders and high levels of repetitive behavior: citalopram ineffective in children with autism. Arch Gen Psychiatry 2009, 66(6):583-590.
Competing interests
Authors of original article
ppO2 seems to be the key factor
13 April 2010
Presuming for the sake of discussion that treatment is effective.
As the grandparent of an autistic child and a former commercial diver I have great personal interest in this subject, and knowledge of diving gases and physiological effects.
After a serious Internet search, I can find no evidence, nor any serious claim, of any proposed mechanism for these effects EXCEPT for the increased partial pressure of O2 (ppO2).
Given that an oxygen concentrator can produce 95% O2 (perhaps as low as 80% actually delivered) and that the treatment dives of this study are only to 1.3 ATA -- or in home devices up to 1.6 ATA -- then we need to seriously consider the idea of treatment with O2 at standard pressure.
1.3 - 1.6 with only slightly increased O2 percentages (e.g, 24%) is MUCH lower than treatment with 80-95% O2 outside of a chamber.
Now clearly using 95% O2 AND 1.6 ATA chamber pressure MIGHT provide additional benefits over O2 alone, but for the cost, time (two adults, a chamber operator AND a companion for the child), slightly increased risks, lowered convenience, etc, we need to know if O2 alone is sufficient/effective/equivalent.
80-95% O2 will provide 4-5 times the normal ppO2, while the study protocols were doing well to reach 1.5 times.
Home or small clinic chambers with O2 might provide about 7 times the ppO2, but are the disadvantages worth the extra cost and trouble?
We have an effective upper limit of 100% O2 and 2 ATA pressure, giving 2.0 ppO2 or roughly a 10 times increase over ambient air.
This is due to both the design of small treatment chambers and the significant increase in central nervous system O2 toxicity beyond 2 ATA ppO2.
Is 0.90 ATA ppO2 effective? More effective than .3 ATA or so?
Is 1.4 pp02 necessary? What is the minimum that gives most of the benefits? How close to 2.0 is most effective?
A $2000 (or less) O2 concentrator is MUCH EASIER to afford and deal with than a $15,000-20,000 chamber (including a concentrator) and much more portable.
Also, this would make it even easier for clinics to offer treatment -- one tech could likely monitor several pairs of child-parent patients, instead of requiring full commitment of attention and a chamber to just one set.
[Of course a much larger chamber could allow for multiple patient sets to dive, but then there are other issues involved.]
We all want to HELP these children -- these costs are real, but trivial if the benefits are real.
Competing interests
My grandson is autistic and a patient of Dr. Rossignol, my grandson anecdotally seems to benefit from the compression treatments -- I have made 5 dives with him myself, and his parents spent significant money on both in clinic treatments and on a home chamber.