Robustness of the healthcare utilization results from the Rotavirus Efficacy and Safety Trial (REST) evaluating the human-bovine (WC3) reassortant pentavalent rotavirus vaccine (RV5)

Background The Rotavirus Efficacy and Safety Trial was a placebo-controlled Phase III study that evaluated the safety and efficacy of a three-dose pentavalent rotavirus vaccine (RV5) including its effect on healthcare utilization for rotavirus gastroenteritis (RVGE). The per-protocol (PP) analyses, which counted events occurring 14 days after dose 3 among infants without protocol violations, have already been published. This paper evaluates the consistency of the healthcare utilization results based on the modified intention to treat (MITT) analyses with the PP analyses. The MITT analyses include all infants receiving at least one dose of vaccine or placebo and follow-up begins after dose 1. The paper also explores the consistency of the results for different subgroups of the study population with different types of surveillance. Methods Data on healthcare utilization for acute gastroenteritis were collected via telephone interviews after administration of the first dose. Parents were either contacted every 6 weeks or every 2 weeks depending on the substudy in which they were enrolled. Those contacted every 2 weeks were also asked to complete symptom diaries. Poisson regression was used to evaluate the effect of RV5 on the rates of RVGE-associated healthcare encounters in all of the analyses. Results In the first 2 years after vaccination, RV5 reduced the combined rate of hospitalizations and emergency department (ED) visits 88.9% (95% CI: 84.9, 91.9) for all RVGE regardless of serotype in the MITT analysis compared with a 94.5% (95% CI: 91.2, 96.6) reduction based on the G1-G4 PP analysis. By type of surveillance, the rate reductions for the G1-G4 PP analysis were 91.0% (95% CI: 81.7, 95.5) and 95.9% (95% CI: 92.2, 97.8) among parents contacted every 2 weeks (number evaluable = 4,451) and every 6 weeks (number evaluable = 52,683) respectively. Conclusions Our analyses demonstrated that the effect of RV5 on reducing the rate of hospitalizations and ED visits based on the MITT analyses were generally consistent with the PP analyses. The rate of events for subgroups with different intensities of surveillance differed but the effect of RV5 on the relative rate reductions were consistent with the results that have already been published. Trial Registration ClinicalTrials.gov number, NCT00090233


Background
Prior to the introduction of rotavirus vaccines, rotavirus gastroenteritis (RVGE) was the most common cause of severe gastroenteritis in infants and young children. Almost every child has been infected with rotavirus by 5 years of age in both developing and industrialized countries [1]. Rotavirus-positive acute gastroenteritis is more severe than rotavirus-negative acute gastroenteritis with higher rates of dehydration, vomiting, fever, and signs of lethargy [2]. There are nearly 600,000 deaths due to RVGE worldwide which mainly occur in developing countries [1]. In industrialized countries, hospitalization rates had been high prior to the introduction of vaccination programs [3]. Children are usually admitted to the hospital to prevent or treat dehydration [4][5][6]. RVGE most often occurs at specific times of the year [7]. In the United States, for example, rotavirus activity follows a distinct winter-spring seasonal pattern [8].
Since RV5 was included in the immunization schedule in the U.S., surveillance studies have been conducted to evaluate vaccine effectiveness. In a national sentinel network of laboratories, the median length of the rotavirus season decreased by 54%, the number of stool specimens submitted for testing decreased by 67%, and the proportion of test results positive for rotavirus declined by 69% after vaccination in the 2007-8 season compared to the 2000-2006 seasons [15]. Similar results were reported at Children's Hospital of Philadelphia [16]. In the 2007-8 seasons, there was an 87% decline in community acquired rotavirus cases compared to the same time period in [2005][2006]. Other studies reported sharp reductions in rotavirus-related hospitalizations and emergency department (ED) visits after RV5 became available even among age groups that would not have been eligible to receive the vaccine [17][18][19]. These reductions in healthcare utilization parallel what was observed in the Phase III trial.
The efficacy and safety of RV5 was evaluated in the Rotavirus Efficacy and Safety Trial (REST), a large international placebo-controlled, blinded trial conducted from 2001 to 2005 with nearly 70,000 healthy infants enrolled from 11 countries. Infants were to receive three doses of either RV5 or placebo. The large sample size and the length of follow-up in REST provided a unique opportunity to quantify the effect of RV5 on healthcare outcomes related to RVGE in a prelicensure study. In the per-protocol (PP) analysis, which was comprised of infants without protocol violations and counted events occurring 14 days after dose 3, RV5 reduced the rate of hospitalizations and ED visits related to G1-G4 RVGE by 94.5% (95% CI: 91.2 -96.6) in the first two years after vaccination [10]. The rate reductions were generally consistent among study populations across three regions -the US (including the Navajo and White Mountain Apache Nations), Europe, and Latin America [20].
The efficacy reported in clinical trials can not always be directly extrapolated to other populations because of the strict inclusion criteria for enrollment as well as the close monitoring of subjects in the trial [21,22], especially when the results are based on a PP analysis. This paper assesses the robustness of the previously published RVGE-associated healthcare utilization results from REST based on the PP analysis by evaluating the reduction in RVGE-associated healthcare utilization regardless of serotype in a modified intention-to-treat (MITT) analysis among infants receiving at least one dose of vaccine. The method and frequency of surveillance also differed for specific subgroups within the study population. The effects of the differences in intensity of surveillance on healthcare utilization rates were also explored.

Study design and case definitions
In REST, infants were randomly assigned in a 1:1 ratio to receive either vaccine or visibly indistinguishable placebo starting at 6-12 weeks of age, followed by two more doses at 4-10 week intervals up to 32 weeks of age [10,20]. Doses were administered year-round without regard to RV seasonality. The case definition for acute gastroenteritis (AGE) was 3 or more watery or looser-than-normal stools within a 24-hour period and/or forceful vomiting along with the identification of rotavirus by enzyme immunoassay (EIA) in a stool specimen taken within 14 days after the onset of symptoms. G-serotypes were identified by one-step reverse-transcriptase-polymerase chain reaction (PCR) amplification followed by sequencing [23]. More complete detail about the study design and the methods used to analyze the data were provided in the earlier publications [10,20]. The protocol was approved by the ethical review committee at each site and conducted in conformance with applicable country or local requirements. All guardians signed informed consent prior to vaccination.

PP vs. MITT analyses
Earlier publications have presented the PP analyses for the effect of RV5 on reducing the rates of hospitalizations and/or ED visits for the overall study population as well as the rate reductions by region and serotype for the first two years after vaccination. The analyses for the overall study population and the analyses stratified by region were based on the G1, G2, G3, and G4 serotypes included in RV5. The effect of RV5 on hospitalizations and ED vis-its combined was also determined individually for each of the 5 most common G-serotypes circulating in Europe and the Americas: G1, G2, G3, G4 and G9.
For the analyses stratified by region, three separate regions were examined: the US including the Navajo and White Mountain Apache Nations (but not Puerto Rico), Europe, and Latin America or the Caribbean including the Commonwealth of Puerto Rico [20]. Although a selfgoverning island that is part of the commonwealth of the United States, Puerto Rico was included in the Latin American/Caribbean region for these analyses because of its close proximity to the Caribbean.
In this paper we compare the G1-G4 PP analyses with the MITT analyses regardless of serotype based on infants receiving at least one dose of vaccine or placebo where all healthcare encounters anytime after the first dose are counted for the first two years after vaccination. The analyses for the overall study population as well as the analyses stratified by region include all episodes of RVGE identified by EIA regardless of G-serotypes including episodes where rotavirus antigen was detected by enzyme immunoassay but the serotype could not be determined by PCR. Table 1 provides a detailed comparison of the inclusion criteria for the PP and MITT analyses. Other variations of the PP and MITT analyses were also presented in order to assess the potential effect of including all RVGE serotypes, subjects with protocol violations, and the protection prior to completion of the vaccine regimen for the overall study population.

Method and frequency of surveillance
Several substudies were nested within REST. The surveillance method and the frequency of contact with parents differed between the Clinical Efficacy substudy and the remainder of the REST cohort. Data on healthcare outcomes was collected via telephone interviews with parents for all infants. Parents were contacted on days 7, 14, and 42 after each dose. The parents of infants in the Clinical Efficacy substudy were then contacted every two weeks until their children completed the study whereas the parents of the infants not in the substudy were contacted every 6 weeks. Health Care Utilization Questionnaires were completed by study personnel during the interviews. For infants participating at sites included in the substudy, all episodes of AGE were reported and parents completed a daily Acute Gastroenteritis Report Card which permitted an assessment of the severity of each acute gastroenteritis episode. For infants not included in the Clinical Efficacy substudy, only hospitalizations and ED visits associated with episodes of AGE were reported and no diary cards were completed. This paper also explores how the intensity of surveillance affected the reported rates of healthcare encounters for RVGE and the observed relative reduction in the rate of RVGE health-care encounters by comparing the results at study sites included in the Clinical Efficacy substudy with all other sites.
Given that there were some AGE episodes where stool specimens were not collected, all AGE episodes were also analyzed to determine whether any differences between the two subgroups with respect to AGE/RVGE between subjects enrolled in the Clinical Efficacy substudy and other subjects in REST were due to differences in the rates of healthcare encounters for AGE reported or differences in the ability to collect stool specimens and confirm whether the encounters were associated with RVGE.

Statistical analyses
Poisson regression was used to compare the rates of healthcare encounters in the vaccine and placebo groups. For the analyses based on the overall study population and by serotype [10], generalized estimating equations (GEE) were used to adjust the standard errors and account for correlations between observations [24,25]. GEE is not optimal when the count of events is small [26]. However, because this methodology was applied to the serotype specific PP analyses in an earlier publication, it is applied here to the MITT analyses to be consistent. GEE also cannot be used when there are no healthcare encounters in the vaccine or placebo groups. Therefore, the exact binomial method for ratios of Poisson counts was used for the analyses stratified by region as well as for analyses where there was a zero count in one of the groups being compared. The rates of healthcare encounters were expressed as the annual number of encounters per 1000 person-years because the length of follow-up differed among infants enrolled in the study. All infants were to be followed for at least 42 days after the last vaccination and up to 365 days after the first dose or until the end of the study, whichever came first. A subset of infants in the Clinical Efficacy substudy enrolled during a rotavirus season was followed until the end of the next rotavirus season which resulted in follow-up for time periods for a maximum of 2 years for some infants.

Infants included in the MITT and PP analyses
Data for 69,274 randomly assigned infants were available in the REST clinical database at the time the Data and Safety Monitoring Board determined that the study had satisfied its criterion for stopping enrollment associated with the primary safety endpoint assessing the risk of intussception among vaccine recipients. After that date, an additional 1027 infants were only followed for safety but not efficacy. In total, 68,038 infants received at least one dose of vaccine or placebo including 5,673 vaccinated infants in the efficacy substudy ( Figure 1). The PP analyses included 28,646 evaluable infants in the RV5 group and 28,488 evaluable infants in the placebo group. There were 9,518 infants (4,740 RV5; 4,778 Placebo) with protocol violations and 8,773 infants (4,367 RV5; 4,406 Placebo) violated the protocol because they did not receive all 3 doses.
The MITT analysis included 33,163 evaluable infants in the RV5 group and 33,015 evaluable infants in the placebo group. The MITT analysis differed from a strict intention-to-treat (ITT) population because it excluded infants who were randomized but never vaccinated since these infants were not followed for efficacy outcomes. Figure 2 accounts for all of the infants excluded from the MITT analysis in the RV5 and placebo groups with the reasons for exclusion. Only 1.8% of the infants randomized were excluded from the MITT analysis because they were never vaccinated. Of the 1,860 infants receiving at least one dose of vaccine or placebo excluded from the MITT analysis, 92.3% reported an episode of acute gastroenteritis but either no stool specimens were collected to determine whether they were positive for rotavirus or the stool specimens were collected outside the allowed time period. Table 2 provides a comparison of the baseline characteristics for age, race, and gender based on the number of evaluable subjects for the G1-G4 PP analysis and the MITT analysis regardless of serotype.

Comparison of results based on PP analysis for serotypes G1-G4 and PP analysis regardless of serotype for overall study population
There were 389 hospitalizations and emergency department (ED) visits (20 RV5; 369 Placebo) in the PP analysis limited to the G1-G4 serotypes (Table 3). When all episodes of rotavirus gastroenteritis regardless of serotype were included in the PP analysis (with follow-up beginning 14 days after administration of the third dose), there were 429 hospitalizations and ED visits (22 RV5, 407 Pdata not shown in tables) and RV5 reduced the rate of hospitalizations and ED visits for the first 2 years after vaccination by 94.6% (95% CI: 91.5, 96.5) for the PP analysis regardless of serotype compared with a 94.5% (95% CI: 91.2, 96.6) rate reduction based on the PP analysis for the G1-G4 serotypes. In addition to the 389 hospitalizations and ED visits included in the PP analysis limited to the G1-G4 serotypes, the PP analysis regardless of serotype also included 14 (0 RV5, 14 P) hospitalizations and ED visits associated with the G9 serotype, 1 ED visit associated with the G12 serotype, and 25 (2 RV5, 23 P) hospitalizations and ED visits where the serotype could not be identified by PCR.

Comparison of results based on PP analysis regardless of serotype and MITT analysis regardless of serotype for overall study population
When all episodes of rotavirus regardless of serotype were included and follow-up began after administration of the first dose of RV5 or placebo in the MITT analysis, there were 580 (58 RV5, 522 P) hospitalizations and ED visits included. RV5 reduced the rate of hospitalizations and ED visits by 88.9% (95% CI: 84.9, 91.9). There were 151 additional hospitalizations and ED visits in the MITT analysis compared with the PP analysis regardless of serotype. Of this total, 43 hospitalizations and ED visits (14 RV5, 29 Placebo) occurred among protocol violators and 107 hospitalizations and ED visits (21 RV5, 86 Placebo) occurred prior to 14 days after dose 3 of which 22 hospitalizations and ED visits (12 RV5, 10 Placebo) occurred within the 13 days after receiving the first dose. There was also one additional ED visit that occurred more than 14 days after dose 3 in an RV5 recipient classified as not evaluable in the PP analysis. If the MITT analysis excluded protocol violators, the reduction in the rate of hospitalizations and emergency department visits associated with RV5 would be 91.0% (95% CI: 87.3, 93.5 -data not shown in tables). If the MITT analysis excluded protocol violators and only counted RVGE-related hospitalizations and ED visits beginning 14 days after dose 1, the     6, 89.3) a Subjects who received a vial where a temperature excursion occurred, subjects who had less than 3 vaccinations or less than 28 days between vaccinations, and subjects who were cross-treated or prematurely unblinded b In the PP analysis follow-up begins 14 days after dose 3; in the MITT analysis follow-up begins after dose 1 c The RV5 and Placebo columns represent the number (rate) of events; the rates reflect the incidence density expressed as the annual number of events rate reduction would be 93.3% (95% CI: 90.2, 95.4 -data not shown in tables).

Comparison of PP and MITT analyses stratified by serotype
The effect of RV5 on the rate of RVGE hospitalizations and ED visits combined for each of 5 common serotypes currently circulating in the Americas and Europe identified during REST are shown for the MITT and PP analyses in Table 4. Compared with the PP analysis, the number of hospitalizations and ED visits in the MITT analysis increased by 102 (16 RV5, 86 P) for the G1 serotype, 4 (0 RV5, 4 P) for the G2 serotype, 7 (2 RV5, 5 P) for the G3 serotype, 2 (0 RV5, 2 P) for the G4 serotype, and 13 (2 RV5, 11 P) for the G9 serotype. The rate reduction for the G2 serotype, 87.6% (95% CI: <0, 98.5) was not statistically significant for the PP analysis. However, the 91.7% (95% CI: 34.7, 99.0) rate reduction reflected better sensitivity of the MITT analysis to the difference between RV5 and placebo for G2. The rate reductions for the G1, G3, G4 and G9 serotypes were statistically significant based on both the PP and MITT analyses. By serotype, the rate reductions were 95.1% (95% CI: 91. 6 Table 6 presents the effect of RV5 on the rates of hospitalizations and ED visits for RVGE and AGE as well as the number and rate of events (annual number of events per 1000 person years) among infants in the RV5 and placebo groups for the PP analysis stratified by the intensity of surveillance at the study site. There were 4,451 infants in the Clinical Efficacy substudy (2,173 RV5; 2,278 Placebo) and the remaining 52,683 infants were participating at other sites (26,473 RV5;26,210 Placebo -numbers not shown in tables). The relative reduction in the rate of hospitalizations and emergency department visits for RVGE and AGE in the Clinical Efficacy substudy were generally consistent with the rate reductions at other sites. By type of surveillance, the rate reductions were 91.0% (95% CI: 81.7, 95.5) for the G1-G4 PP analysis and 39.7% (95% CI: 28.0, 49.5) for all AGE among parents contacted every 2 weeks compared with 95.9% (95% CI: 92.2, 97.8) for the G1-G4 PP analysis and 41.2% (95% CI: 35.0, 46.8) for all AGE among parents contacted every 6 weeks. However, the rate of G1-G4 RVGE hospitalizations and ED visits reported among infants in the placebo group in the Clinical Efficacy substudy (51.8 per 1000 person years) was more than three times the rate for placebo recipients not in the substudy (15.6 per 1000 person years). The difference in the rates for ED visits was more pronounced than the differences in the rates for hospitalizations. Similarly, the rate of hospitalizations and ED visits for AGE reported among infants in the placebo group in the Clinical Efficacy substudy (131.4 per 1000 person years) was nearly 1.5 times higher than the rate among infants for RVGE in the placebo group for infants not in the Clinical Efficacy substudy (53.8 per 1000 person years).

Discussion
The results for REST presented here demonstrate that the reduction in the rates of hospitalizations and ED visits associated with the use of RV5 remained very high in the MITT analysis regardless of serotype despite small differences between the G1-G4 PP analysis and the MITT analysis regardless of serotype. The MITT analysis was more inclusive than the PP analysis because it included protocol violators, and RVGE episodes regardless of serotype immediately following dose 1. Of the 69,274 randomized infants enrolled in REST, only 3,096 (4.5%) were excluded from the MITT analysis of whom 1481 were in the vaccine group and 1615 were in the placebo group. Among those excluded who received at least one dose of vaccine, the majority did not have stool specimens collected to determine whether the episodes were attribut- a Subjects who received a vial where a temperature excursion occurred, subjects who had less than 3 vaccinations or less than 28 days between vaccinations, and subjects who were cross-treated or prematurely unblinded b In the PP analysis follow-up begins 14 days after dose 3; in the MITT analysis follow-up begins after dose 1 c The numbers in the RV5 and Placebo columns represent the number (rate) of events; the rates reflect the incidence density expressed as the annual number of events per 1000 person-years d The number of hospitalizations and ED visits among placebo recipients for the per-protocol population has been revised since the NEJM publication. One additional healthcare encounter in the placebo group was identified after the NEJM publication [10]    able to RVGE or they did not have evaluable follow-up time. The small differences between the G1-G4 PP analysis and the MITT analysis regardless of serotype for the overall study population appeared to be associated with the inclusion of protocol violators in the MITT analysis and the time needed for an immune response to develop within the first 14 days after dose 1. There was no difference in the reduction in the rate of hospitalizations and ED visits for the G1-G4 PP analysis and the PP analysis regardless of serotype. High efficacy has been established with post hoc analyses from 14 days after dose 1 until 14 days after dose 3 [27].
The consistency of the results for the PP and MITT analyses extended to other subgroups within the study population. The comparison of the PP and MITT analyses for premature infants has already been published [28]. The reduction in the rate of hospitalizations and ED visits was 100% (95% CI: 74.0, 100) for the G1-G4 PP analysis and 95.5% (95% CI: 76.4, 99.9) for the MITT analysis regardless of serotype among premature infants [28].
The higher number of healthcare encounters included in the MITT analysis provided better precision for the reductions in hospitalizations and ED visits combined for each of the most prevalent serotypes circulating in the US, Europe, and Latin America, and the broader scope of the MITT analysis enhanced the generalizability of such results. The rate reduction for the G2 serotype based on the MITT analysis also provided better sensitivity to the difference between RV5 and placebo for G2.
When the analyses were stratified by region, the rates of healthcare utilization among infants in the placebo group differed across geographic areas, which may be explained by differences in healthcare systems. However, the relative effect of RV5 on reducing these rates was generally consistent across regions based on the PP analysis [20]. The rate reductions for the U.S. and the Latin American/ Caribbean regions based on the MITT analysis regardless of serotype were somewhat lower than the G1-G4 PP analyses. In the US the difference in the rate reductions for the G1-G4 PP analysis and the MITT analysis regardless of serotype was more pronounced for ED visits. The rate reductions for hospitalizations were 100% (95% CI: 73.8, 100) based on the G1-G4 PP analysis compared with 97.4% (95% CI: 84.8, 99.9) based on the MITT analysis regardless of serotype.
When the study population was stratified based on the intensity of surveillance, the rates of healthcare encounters among infants in the placebo group differed but the The numbers in the RV5 and Placebo columns represent the number (rate) of events; the rates reflect the incidence density expressed as the annual number of events per 1000 person years impact of the vaccine on reducing the rates of healthcare encounters was consistent for infants included in the Clinical Efficacy substudy compared with other infants. The difference in the rates of hospitalizations and ED visits for RVGE based on the intensity of surveillance likely reflects a combination of better recall of events among parents of infants who were monitored more closely as well as the higher proportion of stool specimens collected. The efficacy of RV5 against RVGE of any severity was also evaluated in the Clinical Efficacy substudy. These results were reported in an earlier publication for the PP and MITT analyses based on the G1-G4 serotypes [10,27]. In the first rotavirus season after vaccination the efficacy against G1-G4 RVGE regardless of severity was 74% (95% CI: 66.8, 79.9) and 60% (95% CI: 57.5, 67.1) for the PP and MITT analyses respectively [10]. When all RVGE regardless of serotype were included, the efficacy was 71.8% (95% CI: 64.5, 77.8 -data not shown in tables) and 50.9% (95% CI: 41.6, 58.9 -data not shown in tables) for the PP and MITT analyses respectively.

Conclusion
Our analyses demonstrate the robust and consistent performance of the vaccine and its relevance for infants and young children in a prelicensure study. These findings reflect the generalizability of the effect of RV5 on reducing RVGE including the healthcare burden associated with serotypes not included in the vaccine and was evident when follow-up began immediately following the first dose. Furthermore, the results did not depend on the location of the sites or the intensity of the surveillance. Parents of children in the Clinical Efficacy substudy were contacted more frequently compared to other parents and were more likely to recall events. This increased the power of the study to detect differences between the vaccine and the placebo groups with a smaller sample size but it did not influence the observed relative effect of RV5 on reducing the healthcare burden. This observation should reassure researchers designing future clinical trials for rotavirus gastroenteritis that the time interval chosen to collect healthcare utilization data should not bias the results but it may affect the power of the study.