Randomized controlled trials (RCTs) that use the modified intention-to-treat (mITT) approach are increasingly being published. Such trials have a preponderance of post-randomization exclusions, industry sponsorship, and favourable findings, and little is known whether in terms of these items mITT trials are different with respect to trials that report a standard intention-to-treat.
To determine differences in the methodological quality, sponsorship, authors' conflicts of interest, and findings among trials with different "types" of intention-to-treat, we undertook a cross-sectional study of RCTs published in 2006 in three general medical journals (the Journal of the American Medical Association, the New England Journal of Medicine and the Lancet) and three specialty journals (Antimicrobial Agents and Chemotherapy, the American Heart Journal and the Journal of Clinical Oncology). Trials were categorized based on the "type" of intention-to-treat reporting as follows: ITT, trials reporting the use of standard ITT approach; mITT, trials reporting the use of a "modified intention-to-treat" approach; and "no ITT", trials not reporting the use of any intention-to-treat approach. Two pairs of reviewers independently extracted the data in duplicate. The strength of the associations between the "type" of intention-to-treat reporting and the quality of reporting (sample size calculation, flow-chart, lost to follow-up), the methodological quality of the trials (sequence generation, allocation concealment, and blinding), the funding source, and the findings was determined. Odds ratios (OR) were calculated with 95% confidence intervals (CI).
Of the 367 RCTs included, 197 were classified as ITT, 56 as mITT, and 114 as "no ITT" trials. The quality of reporting and the methodological quality of the mITT trials were similar to those of the ITT trials; however, the mITT trials were more likely to report post-randomization exclusions (adjusted OR 3.43 [95%CI, 1.70 to 6.95]; P < 0.001). We found a strong association between trials classified as mITT and for-profit agency sponsorship (adjusted OR 7.41 [95%CI, 3.14 to 17.48]; P < .001) as well as the presence of authors' conflicts of interest (adjusted OR 5.14 [95%CI, 2.12 to 12.48]; P < .001). There was no association between mITT reporting and favourable results; in general, however, trials with for-profit agency sponsorship were significantly associated with favourable results (adjusted OR 2.30; [95%CI, 1.28 to 4.16]; P = 0.006).
We found that the mITT trials were significantly more likely to perform post-randomization exclusions and were strongly associated with industry funding and authors' conflicts of interest.
The intention-to-treat principle requires that all participants that are randomized must be included in the final analysis and analyzed according to the treatment group to which they were originally assigned, regardless of the treatment received, withdrawals, lost to follow-up or cross-overs. Despite this principle, in many instances in randomized trials the term intention-to-treat was inappropriately described and participants improperly excluded[1-4]. In addition, the use of a modified intention-to-treat (mITT) approach in randomized controlled trials (RCTs) is increasingly appearing in the medical literature according to a systematic review. There is no clear definition of what is mITT. In fact, descriptions of mITT analyses vary greatly from trial to trial, and often contain more than one criterion that are difficult to interpret as it is not easy to discriminate between missing data cases and deviations from protocol. Moreover, while post-randomization exclusion appear to be the primary factor that characterizes the mITT analysis, the majority of the trials were industry sponsored and reported results that favoured the therapy under investigation. The uncertainty or equipoise principle that should exist in the design of RCTs states that over time, the mean benefit of investigational therapies and comparison therapies should be equal . The high prevalence of significant results reported in randomized trials might not reflect true differences. Some argue that this principle can be violated in the presence of financial ties between investigators and the pharmaceutical industry . The biomedical literature reports strong and consistent evidence that industry-sponsored research is likely to produce pro-industry conclusions . The appearance of mITT reporting in modern RCTs could be a consequence of the widespread financial ties that exists among investigators and industry.
In this cross-sectional study, we examined whether mITT reporting trials were different in terms of methodological quality from trials with other types of intention-to-treat reporting. Our first hypothesis was that studies using mITT analyses would be associated with limited quality of reporting and study characteristics based on the assumption that mITT reporting is a limitation in principle. Moreover, we investigated whether the reporting of mITT in RCTs was associated with industry sponsorship, the presence of authors' conflicts of interest, and favourable findings.
Selection of Studies
To compare methodological quality, industry sponsorship, the presence of authors' conflicts of interest, and findings among trials based on the type of intention-to-treat reporting, we sought to identify the top medical journals and specialty journals published in 2006 that were more likely to report the use of an mITT approach according to our previous survey . The three high-ranking medical journals were the Journal of American Medical Association, the New England Journal of Medicine and the Lancet; and the three specialty journals were Antimicrobial Agents and Chemotherapy, the American Heart Journal and the Journal of Clinical Oncology. We carried out a computerized search in Medline (via PubMed) to identify RCTs published in these six journals using the publication type "randomized controlled trials". The results were then cross-checked against a search of the Cochrane Central Register of Controlled Trials.
Excluded reports included: reports of phase I trials; pharmacokinetic, pharmacodymanic or dose-comparing studies; cluster RCTs; post-hoc studies; research letters; cost-effectiveness studies; study protocols; and prognostic and diagnostic studies. Two authors screened all of the full-text journal articles to confirm their eligibility for inclusion and excluded 97 articles (Figure 1).
Figure 1. Study Screening Process. AAC: Antimicrobial Agents and Chemotherapy; AHJ: American Heart Journal; JAMA: Journal of American Medical Association; JCO: Journal of Clinical Oncology; NEJM: New England Journal of Medicine.
Two other reviewers independently and in duplicate, extracted the following information from the journal articles: characteristics of the trial, primary outcomes, number of allocation groups, number of patients, main outcome measure, P-values, and number of subjects excluded from the analysis. Moreover, reporting of flow-charts, sample size calculations, and information regarding missing data, withdrawals or patients lost to follow-up were recorded.
The relevant RCTs were subsequently classified according to the type of intention-to-treat analyses used as follows: ITT, trials reporting the use of standard ITT analyses; mITT, trials reporting the use of "modified intention-to-treat" analyses; or "no ITT" trials not reporting the use of any intention-to-treat analyses. This classification was independent of the reporting of any post-randomisation exclusion. Trials reporting the use of ITT with descriptions or conditions different from the standard intention-to-treat definition were classified as mITT.
Descriptions of mITT were retrieved to evaluate the type and deviation from a true intention-to-treat analysis according to our previous classification.
Assessment of Methodological Quality
Sequence generation, allocation concealment and blinding were used as indicators of methodological quality and were classified as "adequate," "inadequate" or "unclear." In our analysis, the "inadequate" and "unclear" categories were collapsed into "not adequate." For sequence generation, the following approaches were considered adequate: random number table, computer random number generator, drawing of lots and envelopes. The use of methods such as case record number, date of birth or day, month, or year of admission were considered inadequate for sequence generation.
For allocation concealment, central randomisation, coded drug packs prepared by an independent pharmacist and sequentially numbered, sealed and opaque envelopes or any method that hindered the researchers' ability to foresee the next assignment was considered adequate. Methods such as procedures based on inadequate generation of allocation sequences, open allocation schedules, alternation and unsealed or non-opaque envelopes were considered inadequate.
Regarding blinding, a trial was considered adequately blinded if it was described as "double-blinded" and used adequate methods such as identical placebo tablets or adequate descriptions of who was blinded (e.g., the outcome assessor) in cases where blinding of participants and caregivers may not be feasible. Each eligible trial was independently reviewed for methodological quality. Discrepancies were resolved by discussion, and a consensus was reached for all trials.
Information regarding the sources of funding for each trial was extracted from the articles by reviewing the article text, the conflict of interest section, information on funding (if present) and the acknowledgements section. RTC funding sources were abstracted and categorized as follows: (1) not-for-profit organization; (2) for-profit agency; (3) co-financed, indicating funding from both not-for-profit organization(s) and for-profit agency(s); (4) no funding; and (5) not reported.
Authors' Potential Conflicts of Interest
The presence or absence of author conflicts of interest was determined by reviewing the authors' institutional affiliations, the conflicts of interest section, information on funding (if present) and the acknowledgements section.
The institutional affiliation of the first author (e.g., university, not-for-profit organization or for-profit agency) was retrieved from each article. The affiliation of the other authors was also recorded when a financial tie with a for-profit agency was present. Subsequently, the studies were classified for analysis as follows: (1) authors declaring no competing interests; (2) financial ties with the sponsor of the study disclosed, indicating that at least one author participated on behalf of a for-profit agency (e.g., as an employee or consultant); (3) other financial ties disclosed, indicating that an author did not participate on behalf of the study sponsor but did report a conflict of interest such as a past financial tie with the pharmaceutical industry; and (4) conflicts of interest not reported.
For each published trial, the results for each primary outcome were classified as one of the following: (1) favourable, if the result was statistically significant with P < 0.05 or if the confidence interval did not include the null; or (2) inconclusive, if the result did not reach statistical significance.
For equivalence or noninferiority studies, the result was coded as favourable when the assumption of equivalence or noninferiority was satisfied.
The distribution of various characteristics of the examined trials such as the type of ITT analysis and the source of funding were reported using descriptive statistics. The Pearson chi-square test was used for contingency table analysis. Interobserver agreement was measured using the kappa (κ) statistic.
We investigated the associations between the "type" of intention-to-treat analysis (as a dependent, multinomial 3-level variable: ITT, mITT, or "no ITT") and the methodological quality (sequence generation, allocation concealment and blinding; Model A), the quality of reporting (reporting of a flow-chart, sample size calculation, and lost to follow-up; Model A), the funding source (Model B), and the presence of authors' conflicts of interest (Model C) using multinomial logistic regression.
Bivariate analyses were performed to identify independent variables with a P value less than or equal to 0.10. These potential confounding variables were included in each model of multivariate multinomial logistic regression analysis. In each model, the variables of journal type, use of placebo, the presence of post-randomisation exclusions, and all of the items used as indicators of methodological quality and quality of reporting were included. In model A, in addition to all the mentioned variables, the use of funding was included (the inclusion of the presence of authors' conflict of interest instead of funding did not produce any change in the results of the model).
In each model, the ITT category was used as a reference for comparisons.
Furthermore, we performed univariate logistic regression to investigate the association between findings (as a dependent variable) and the "type" of intention-to-treat (as an independent polynomial 3-level variable: ITT, mITT, "no ITT"). To account for potential confounding variables we performed a multivariate logistic regression and included in the model the type of the journal, the use of placebo, the presence of post-randomisation exclusions, the use of funding and all the items that were indicators of reporting and methodological quality.
Finally, we assessed the overall association between favourable findings and no for-profit sponsorship using a logistic regression (univariate and multivariate model were used; all of the above mentioned variables including the "type" of intention-to-treat were used for adjustment). This was done to evaluate any consistency with studies in the biomedical literature that report a strong association between industry sponsorship and positive findings. The Pearson goodness-of-fit test was used to assess the overall fit of the models. Odds ratios (ORs) were then computed with confidence intervals (CIs).
A two-tailed P-value less than 0.05 was considered statistically significant. The analyses were performed using STATA/SE, version 8.2 for Windows (StataCorp, College Station, Texas).
Characteristics of the Included Studies
Our final sample consisted of 367 published RCTs. Of these, 197 were classified as ITT trials, 114 as "no ITT" trials and 56 as mITT trials (analysis of number and types of mITT deviations are reported in Appendix 1).
The number of participants included in each RCT ranged from 10 to 160,921 (median, 368; interquartile range 140-991). Trials classified as mITT were significantly more likely to be published in general medical journals, to report post-randomisation exclusions and to use placebo as a comparator. A total of 258 (69%) trials received complete or partial financial support from a for-profit agency and 216 of the trials (60%) reported results that favoured the treatment under investigation. The characteristics of the included trials are shown in Table 1.
Table 1. Characteristics of Included Randomised Controlled Trials.
The kappa value for generation sequence was 0.79 (95% CI 0.74 to 0.85), for allocation concealment 0.82 (95% CI 0.77 to 0.88); for blinding, 0.78 (95% CI 0.72 to 0.84); and for intention-to treat analysis, 0.90 (95% CI 0.89 to 0.92). The kappa value for funding source was 0.96 (95% CI 0.95 to 0.97); for authors' conflict of interest, 0.94 (95% CI 0.93 to 0.96); and for study findings, 0.93 (95% CI 0.93 to 0.95).
Characteristics of the Included Trials and the Type of Intention-to-treat Reporting
Multinomial logistic regression analyses showed that the mITT trials were more likely to have inadequate or unclear sequence generation and adequate blinding compared to the ITT trials. Trials classified as "no ITT" reported low standards of reporting and methodological quality.
In the multivariate analysis, the mITT trials appeared to have substantially similar methodological quality standards compared to the ITT trials; however, the mITT trials remained more likely to report post-randomisation exclusions (adjusted OR 3.43 [95% CI 1.70 to 6.95]; P = 0.001). Trials classified as "no ITT" were significantly more likely to avoid reporting flow-chart, sample size calculation and lost to follow-up. Table 2 shows the unadjusted and adjusted ORs of the strength of association between characteristics of the studies and the type of intention-to-treat reporting.
Table 2. Association Between Characteristics of the Included Randomised Controlled Trials and the Type of Intention-to-treat Reporting
Sponsorship, Conflicts of Interest and the Type of Intention-to-treat Reporting
Compared to the ITT trials, RCTs classified as mITT were more likely to receive sponsorship from a for-profit agency (adjusted OR 7.41 [95% CI 3.14 to 17.48]; P < 0.001) and were more likely to have at least one investigator who was authoring on behalf of the pharmaceutical industry (adjusted OR 5.14 [95%CI, 2.12 to 12.48]; P < 0.001). Interestingly, both the mITT and "no ITT" trials reported a higher odds ratio of authors not reporting any conflicts of interest. Table 3 shows the unadjusted and adjusted ORs of the strength of association between funding and authors' conflicts of interest and the type of intention-to-treat reporting.
Table 3. Association Between Funding and Authors' Conflicts of Interest for the Included Randomised Controlled Trials and the Type of Intention-to-treat Reporting.
Analysis of the Findings
We did not find any association between mITT reporting and favourable results (adjusted OR 1.27 [95% CI 0.62 to 2.61]; P = 0.51). Overall, however, trials with for-profit agency sponsorship were more likely to report favourable results when compared to trials that did not receive any sponsorship from a for-profit agency (adjusted OR 2.30 [95% CI 1.28 to 4.16]; P = 0.006).
In a sample of RCTs, we assessed the associations between the type of intention-to-treat analysis used and study design characteristics, including the source of funding and the favourability of the results. Our first hypothesis that studies using mITT analyses would be associated with limited quality of reporting and study characteristics was refuted. Instead, we found that the mITT trials were methodologically similar to the ITT trials.
Reports from the literature about the methodological quality of industry sponsored trials are controversial. While some authors report that the methodological quality of industry-sponsored RCTs is limited, others report that trials funded by for-profit agencies have similar or better methodological quality than unsupported trials [8,11].
In our investigation, the mITT trials were more likely to perform post-randomisation exclusions compared to both ITT trials and "no ITT" trials. A study published in 1996 reported a paradox that RCTs reporting ITT with sound methodological quality were more likely to perform exclusions . The authors' interpretation of this finding was that studies with low methodological standards may be less likely to report exclusions, even when exclusions actually occurred. In our analysis, although the mITT trials had the highest occurrence of exclusions, we found that the "no ITT" trials were more likely to perform exclusions compared to the ITT trials. It is possible that the standard of reporting changed over time, and we can speculate that allowing any modification of the standard intention-to-treat analysis may have encouraged authors to report exclusions after deciding which "type" of mITT analysis to perform.
Several studies in the medical literature report that post-randomisation exclusions in randomised trials may lead to biased estimates of the treatment effect. Individual patient data analyses of systematic reviews found that the results were in favor to the treatment under investigation when exclusions were not taken into account in the results rather than when a true intention-to-treat was used. A meta-epidemiological study that investigated 14 meta-analyses in osteoarthritis showed that the exclusion of patients from analysis resulted in a biased estimate of the treatment effect . Another study of pharmaceutical industry sponsored trials investigating serotonin reuptake inhibitors found that results were more in favour when a per-protocol analysis was used instead of intention-to-treat analysis .
The second aim of our study was to assess the associations between the type of intention-to-treat analysis used and the presence of industry sponsorship or author conflicts of interest among RCTs. In the RCTs included in this study the use of mITT analysis was strongly associated with industry funding and author conflicts of interest.
In the medical literature, there is plenty of evidence that the pharmaceutical industry is directly or indirectly involved at different stages in the conduct, design and publication of biomedical research[9,11,16]. The selection of an inappropriate comparison group (e.g., a drug with non-equivalent dosage) [8,17,18], multiple reporting of studies [19,20] and suppression or delay of publications  are all circumstances where the presence of industry sponsorship are well documented.
We did not find any association between favourable results and mITT reporting. The number of mITT trials was too low to detect any difference; however, even if the sample of trials was adequate, we are aware that any potential difference cannot be free of bias given the large heterogeneity of the included trials in terms of the intervention and outcome investigated. We believe, nonetheless, that the appropriate place to evaluate the mITT as a source of bias is to explore its impact in meta-epidemiological studies.
In our previous survey, we showed that publications of mITT reporting trials is significantly increasing. For example, the overall incidence of trials published in 2006 in the medical literature was around 5%. This incidence of mITT reporting trials was underestimated since the studies claiming the use of the intention-to-treat approach when in fact they used the mITT approach (owing to the type of deviation present in the description of the analysis) may have escaped the previous search. Indeed, in the present study, we were able to assess all the randomized trials in 6 journals, and capture those trials that reported intention-to-treat but with deviations. The number of these trials (n = 32) was greater than the number of the trials that actually reported as "modified" (n = 24; Box). Consequently, we can conclude that the phenomenon of mITT is wider than previously estimated.
Our main limitation was that we used data from RCTs that were published in six journals; therefore, we are unsure that our results can be generalized to other trials published elsewhere. To test our aforementioned hypothesis, we needed to compare mITT trials with trials reporting (or not reporting) intention-to-treat. Although our previous research has documented an increasing incidence of mITT trials, the proportion of mITT trials being conducted remains too low to obtain an adequate sample for comparison. Consequently, we decided to target journals that were more likely to report mITT trials by selecting general medical and specialty journals.
Data analysis is the crucial final stage of any study design. Well-designed RCTs require strict adherence to the intention-to-treat principle, in which subjects should remain in the group to which they were originally allocated. Although further research is needed to document that the mITT is a potential source of bias, to limit unjustified exclusions, the misuse of "intention-to-treat" terminology should be abandoned, and authors need to adhere to the standard intention-to-treat principle. The updated version of the CONSORT statement goes into this direction and suggests the replacement of any kind of intention-to-treat reporting with a clear description of exactly who was included in each analysis.
Analyses of Types and Deviations from Intention-to-Treat of Trials classified as mITT
Of the 56 trials classified as mITT, 24 were reported explicitly as "modified" while 32 reported "intention-to-treat" but reported descriptions that deviate from the true ITT and thus were considered as mITT trials.
Overall, 31 (55%) trials reported 1 type of mITT deviation, 17 (30%) reported 2 types of mITT deviation, 5 (9%) reported 3 types of mITT deviation, and 3 (5%) did not report any type of mITT deviation. In 35 (63%) trials, the main exclusion criterion for the mITT analysis was treatment-related; in 18 of such trials, the treatment-related mITT was accompanied by at least one additional type of mITT deviation.
In 17 (30%) trials, the exclusion criteria used to justify the mITT criteria was the absence of a post-baseline assessment; in 13 of these trials, it was accompanied by at least one other type of mITT deviation.
Baseline assessment related mITT was described in 10 (18%) trials; in 6 of these the approach was accompanied by another type of deviation. Four (7%) trials required a target condition to describe the mITT deviation; and 1 (2%) trial fell into the follow-up related mITT deviation. While 13 (23%) trials did not fall into none of the above category, 3 (5%) trials remained without any type of mITT description.
Maria Laura Luchetta has been employed in Laboratori Guidotti s.p.a. (Pisa) from November 2003 to October 2006; in Menarini IFR (Firenze) from November 2006 to November 2007. Alessandro Montedori, Maria Isabella Bonacini, Giovanni Casazza, Francesco Cozzolino, Piergiorio Duca, and Iosief Abraha declare that they have no competing interests.
AM and IA conceived and designed the study. AM, MIB, FC, MLL collected data. GC, PD, IA did the statistical analyses. All authors contributed to the interpretation of the results, the writing and critical review of the report. All authors have seen and approved the final version of the report.
We thank the director Paolo Di Loreto, Giuliangela Proietti and Kathryn Mary Mahan for their support.
Health Technol Assess 1998, 2:i-vi.
1-132PubMed Abstract | Publisher Full Text
Ann Intern Med 2010, 152:726-732. PubMed Abstract