Reporting of noninferiority and equivalence randomized trials for major prostaglandins: A systematic survey of the ophthalmology literature

1745-6215-9-69 1745-6215 Methodology Reporting of noninferiority and equivalence randomized trials for major prostaglandins: A systematic survey of the ophthalmology literature Eyawo Oghenowede edeyawo@yahoo.com Lee Chia-Wen chia-Wen.Lee@pfizer.com Rachlis Beth brachlis@hotmail.com Mills J Edward millsej@mcmaster.ca

Faculty of Health Sciences, Simon Fraser University, Burnaby, Canada

Department of Outcomes Research and Evidence Based Medicine, Pfizer Ltd., Surrey, UK

Department of Public Health, University of Toronto, Toronto, Canada

Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Canada

Trials 1745-6215 2008 9 1 69 http://www.trialsjournal.com/content/9/1/69 19055743 10.1186/1745-6215-9-69 10 5 2008 03 12 2008 03 12 2008 2008 Eyawo et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Standards for reporting clinical trials have improved the transparency of patient-important research. The Consolidated Standards of Reporting Trials (CONSORT) published an extension to address noninferiority and equivalence trials. We aimed to determine the reporting quality of prostaglandin noninferiority and equivalence trials in the treatment of glaucoma.

Methods

We searched, independently and in duplicate, 6 electronic databases for eligible trials evaluating prostaglandins. We abstracted data on reporting of methodological criteria, including reporting of per-protocol [PP] and intention-to-treat [ITT] analysis, sample size estimation with margins, type of statistical analysis conducted, efficacy summaries, and use of hyperemia measures.

Results

Trials involving the four major prostaglandin groups (latanoprost, travoprost, bimatoprost, unoprostone) were analyzed. We included 36 noninferiority and 11 equivalence trials. Seventeen out of the included 47 trials (36%, 95% Confidence Intervals [CI]: 24–51) were crossover designs. Only 3 studies (6%, 95% CI: 2–17) reported a presented results of both ITT and PP populations. Twelve studies (26%, 95% CI: 15–39) presented only ITT results but mentioned that PP population had similar results. Thirteen trials (28%, 95% CI: 17–42) presented only PP results with no mention of ITT population results while 17 studies (36%, 95% CI: 24–51) presented only ITT results with no mention of PP population results. Thirty-four (72%, 95% CI: 58–83) of studies adequately described their margin of noninferiority/equivalence. Sequence generation was reported in 22/47 trials (47%, 95% CI: 33–61). Allocation concealment was reported in only 10/47 (21%, 95% CI: 12–35) of the trials. Thirty-five studies (74%, 95% CI: 60–85) employed masking of at least two groups, 4/47 (9%, 95% CI: 3–20) masked only patients and 8/47 (17%, 95% CI: 9–30) were open label studies. Eight (17%, 95% CI: 9–30) of the 47 trials employed a combined test of noninferiority and superiority. We also found 6 differing methods of evaluating hyperemia.

Conclusion

The quality of reporting noninferiority/equivalency trials in the field of glaucoma is markedly heterogeneous. The adoption of the extended CONSORT statement by journals will potentially improve the transparency of this field.

Background

There is emerging debate about the adherence to standards or recommendations made by regulatory bodies in the design, statistical analysis, description, and reporting of randomised clinical trials (RCT) intended to show noninferiority or equivalence 1. The recent extension of the Consolidated Standards of Reporting Trials (CONSORT) statement to noninferiority and equivalence trials 1, the U. S. Food and Drug Administration (FDA) guidelines 2, as well as statements from other regulatory authorities, have updated recommendations to guide the conduct and reporting of RCTs investigating noninferiority or equivalence 3.

Noninferiority and equivalence studies aim to demonstrate that a new experimental treatment is not statistically worse than the active control intervention by more than a pre-specified amount (Δ), the noninferiority or equivalence margin. There are now a large number of studies intending to demonstrate noninferiority or equivalence. However, only very few of them have been designed, described, and reported as such 4. In designs intending to show noninferiority (i.e. that a new treatment is at least as good as or no worse than an existing treatment) or equivalence (i.e. show that a new treatment has equal or comparable efficacy effects), one area of debate has been the application of the intent-to-treat (ITT) principle versus the per-protocol (PP) analysis 5. There has been an expectation that ITT analyses are less conservative, tending to reduce the observed treatment difference in noninferiority trials, thus permitting wider confidence intervals; a factor that has increased the dependence on per-protocol analysis. The International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) E9 guideline on 'statistical principles for clinical trials' unequivocally states that the use of the ITT analysis in equivalence and noninferiority trials is 'generally not conservative and its role should be considered very carefully' 6. However, the Committee for Proprietary Medicinal Products (CPMP) consider both analyses to be equally important 7 and simulation studies have not demonstrated the anticonservative approach of ITT versus PP 8. In the extension of the CONSORT statement, conducting and reporting PP analyses alongside ITT has been suggested as a means of increasing confidence in the study finding particularly when both methods produce similar results.

In order to determine the methodological quality and reporting standards for those trials designed to show noninferiority or therapeutic equivalence, we conducted a systematic survey of the literature to identify noninferiority and equivalence randomised clinical trials involving glaucoma drugs of prostaglandin origin. We applied the extended CONSORT statement for non-inferiority and equivalence trials as methodological criteria of minimum reporting standards 1.

Methods

Eligibility Criteria

Noninferiority and equivalence randomized clinical trials that involved the application of any of the four major drugs of the prostaglandin analogues (latanoprost, travoprost, bimatoprost, and unoprostone) in the treatment of patients with open angle glaucoma (POAG) or ocular hypertension (OH) were eligible for the study. We defined noninferiority trials as 1-sided trials aiming to demonstrate effectiveness not lower than Δ. Whereas, equivalence studies were defined as 2-sided trials wherein the difference between treatments is between -Δ and +Δ margins of equivalence. We further determined a trial to be eligible for inclusion if 'equivalence' or 'noninferiority' was mentioned as the intent anywhere in the methods sections of the manuscripts. We excluded bioequivalence (pharmacokinetic and pharmacodynamic) trials and trials designed to show superiority.

Search Strategy

Using independent and duplicate searchers (OE, EM), we searched the following 6 databases from inception to March 2008: MEDLINE via PubMed, CinAhl, AMED, Toxnet, Cochrane CENTRAL and E-Psyche. Our MEDLINE search strategy identified the exact MeSH terms for the following expressions: open angle glaucoma, ocular hypertension and prostaglandin* (latanoprost, bimatoprost, travoprost aand unoprostone). We conducted 3 main searches. First we collated all articles with the MeSH key terms "glaucoma, open angle OR ocular hypertension" [see Additional file 1]. We then extracted articles containing the MeSH terms "prostaglandin* OR latanoprost OR travoprost OR bimatoprost OR unoprostone." We then pooled these two searches to produce a third list, representing all articles that had as its key terms, "POAG or OH or prostaglandin* OR latanoprost OR travoprost OR bimatoprost OR unoprostone." We also searched multiple databases including CinAhl, AMED, Toxnet, Cochrane CENTRAL and E-Psyche using key search terms like latanoprost, bimatoprost, travoprost, unoprostone, open angle glaucoma, ocular hypertension and prostaglandin*. Simultaneous review of these databases was possible through the use of the Ovid interdisciplinary database that permits the checking of multiple databases. Where full text was unavailable, we searched Google Scholar and also used interlibrary loans to order full text articles. We supplemented this search by reviewing the bibliographies of key papers.

Additional file 1

Example search strategy, MedLine. Search Strategy for MEDLINE via PubMed Search.

Click here for file

Study selection

Working independently and in duplicate OE and EM reviewed all abstracts and full articles, where available, of our search results. We excluded articles at this stage that were not randomized trials or were reviews and/or comments. All full text articles of identified RCT studies that examined treatment with prostaglandin drugs were sought. Eligibility was determined in consensus. Only comparisons containing at least one study drug of interest (i.e., bimatoprost, latanoprost, travoprost or unoprostone) were eligible for inclusion. When comparisons involved several different dosages (e.g., 0.005% or 0.03%), we included only those that met FDA-approved standards. Where we were unsure of specific intentions of noninferiority/equivalence compared to superiority, we contacted one author via email.

Data Collection

We collected information about the type of interventions tested, study design and purpose. We also obtained data on sample size estimation with margins, type of statistical analysis conducted, population analyzed [ITT or/and PP], efficacy summaries, and use of hyperemia measures. Our study evaluation included an assessment of the following general methodological quality features: allocation concealment, sequence generation, masking status, and the handling of losses to follow-up. We also recorded sample size (recruited, randomized and completed) and the use of a combined test of noninferiority and superiority. Finally, we noted whether there was a clear objective/hypothesis relating to noninferiority or equivalence, a tabular presentation of baseline data (demographic and clinical), and a predetermined noninferiority/equivalence margin.

Data Analysis

In order to assess inter-rater reliability on inclusion of articles, we calculated the Phi statistic, which provides a measure of inter-observer agreement independent of chance 9. In order to determine the proportion reporting the specific item with appropriate confidence intervals, we first stabilized the variances of the raw proportions (r/n) using a Freeman-Tukey type arcsine square root transformation: y = arcsine (square root(r/(n+1))+arcsine(square root(r+1)/(n+1))), with a variance of 1/(n+1), where n is the denominator overall size 10. We used StatsDirect (version 2.5.2, http://www.statsdirect.com for all calculations.

Results

Results of literature search

Our first and second search produced 35,207 and 97,523 abstracts respectively. The third and final search (search #1 AND #2 pooled) produced 1144 abstracts. After thorough assessment, 215 abstracts were excluded since they were review articles. Another 549 abstracts were excluded as they were not relevant to present study. Overall, 380 full text papers were retrieved for possible inclusion. Upon careful review of the 380 full text articles, we included 47 full text articles in our analysis (Phi = 0.88). Figure 1 presents details of the exclusion criteria at the various stages during the study selection process.

Figure 1

Flow diagram of included studies

Flow diagram of included studies

Trial interventions

Table 1 outlines the characteristics of all 47 included publications. Of these, 31 reported conducting comparisons of treatments (e.g., travoprost vs. latanoprost) that were duplicated in at least one other included study. More specifically, five trials compared bimatoprost to latanoprost 1112131415. Four trials compared bimatoprost and timolol 16171819. Three trials compared bimatoprost to a fixed combination (FC) treatment that contained timolol 202122.

Table 1

Characteristics of included publications

Author

Year

Sample Size

(N completed)

Treatment Comparison

Topouzis et al 40

2007

332

Travoprost 0.004%/timolol 0.5% versus once-daily latanoprost 0.005%/timolol 0.5%

Hommer 43

2007

430

FC bimatoprost 0.03%/timolol 0.5% versus non-FC bimatoprost 0.03%/timolol 0.5%

Hollo et al 44

2006

192

Timolol maleate versus brinzolamide both added to travoprost

Brittain et al 20

2006

Combination timolol 0.5% and latanoprost 0.005% versus bimatoprost 0.03%

Day et al 21

2005

bimatoprost 0.03% versus timolol maleate 0.5%/dorzolamide 2% FC

Netland et al 45

2003

Brimonidine purite + bimatoprost versus timolol + latanoprost

Thomas et al 28

2003

Latanoprost versus brimonidine

Whitcup et al 16

2003

550

Bimatoprost versus timolol

Netland et al 23

2001

760

Travoprost versus latanoprost versus timolol

Larsson 24

2001

Latanoprost 0.005% versus timolol gel-forming solution 0.5%

Fechtner et al 42

2004

238

FC dorzolamide 2%/timolol 0.5% (COSOPT™) versus latanoprost 0.005% (XALATAN™)

Cohen et al 17

2004

284

Bimatoprost versus timolol

Shin et al 31

2004

242

FC latanoprost/timolol versus FC dorzolamide/timolol

Tomita et al 25

2004

Latanoprost versus timolol

Stewart et al 46

2004

Latanoprost/timolol maleate FC versus concomitant brimonidine and latanoprost therapy

Gandolfi et al 11

2001

214

Bimatoprost versus latanoprost

Watson et al 26

1996

268

Latanoprost versus timolol

Konstas et al 57

2002

Once-daily morning versus evening dosing of concomitant latanoprost/timolol

Day et al 47

2003

Timolol 0.5%/Dorzolamide 2% FC versus timolol maleate 0.5% and unoprostone...

Parrish et al 12

2003

410

Latanoprost versus bimatoprost versus Travoprost

Konstas et al 32

2004

FC latanoprost 0.005%/timolol maleate 0.5% versus FC dorzolamide 2%/timolol maleate 0.5%

Sharpe et al 48

2005

Brimonidine 0.2% versus unoprostone 0.15% both added to timolol maleate 0.5%

Higginbotham et al 18

2002

1009

Bimatoprost versus timolol

Pfeiffer 33

2002

396

FC latanoprost and timolol versus its individual components

Higginbotham et al 34

2002

345

FC latanoprost and timolol versus monotherapy with either components

Konstas et al 49

2008

Latanoprost and dorzolamide/timolol FC after 2 and 6 months of treatment

Konstas et al 37

2007

Latanoprost versus travoprost

Franks et al 39

2006

106

Travoprost 0.004% versus FC latanoprost 0.005%/timolol 0.5%

Diestelhorst et al 35

2006

502

FC latanoprost and timolol in the evening versus concomitant use of the individual components

Hughes et al 41

2005

293

FC travoprost 0.004%/timolol 0.5% ophthalmic solution versus concomitant use of the individual components

Schuman et al 50

2005

387

FC travoprost 0.004%/timolol 0.5% versus concomitant travoprost 0.004% + timolol 0.5 versus timolol BID control

Konstas et al 29

2005

Brimonidine purite versus dorzolamide each added to latanoprost

Diestelhorst et al 36

2004

190

FC latanoprost/timolol versus concomitant latanoprost and timolol

Coleman et al 22

2003

168

Bimatoprost (LUMIGAN) versus combined timolol and dorzolamide (COSOPT)

Zimmerman et al 13

2003

2775

Various POAG monotherapy versus latanoprost

Stewart et al 51

2003

FC timolol maleate/latanoprost versus timolol maleate + brimonidine

Noecker et al 14

2003

249

Bimatoprost versus latanoprost

Honrubia et al 52

2002

217

Latanoprost 0.005% versus FC dorzolamide 2% & timolol 0.5%

Simmons et al 30

2002

107

Brimonidine versus latanoprost in patients uncontrolled on β – blockers

Goldberg et al 38

2001

507

Topical travoprost versus timolol BID

DuBiner et al 15

2001

Bimatoprost versus latanoprost

Brandt et al 19

2001

556

QD, BID bimatoprost versus timolol BID

Susanna et al 53

2001

Latanoprost versus unoprostone

Diestelhorst et al 54

2000

179

Latanoprost versus pilocarpine T.I.D

Stewart et al 55

1999

Timolol solution versus timolol gel each added to latanoprost

Konstas et al 27

1999

Latanoprost versus timolol maleate

Petounis et al 56

2001

148

Latanoprost versus dorzolamide each added to timolol

Six trials compared latanoprost to timolol 132324252627. Three studies compared latanoprost to brimonidine (or brimonidine-containing fixed combination) 282930. Two studies compared FC latanoprost/timolol vs FC dorzolamide/timolol 3132. Two studies compared FC latanoprost/timolol versus its individual components (i.e. latanoprost and timolol) 3334. Two studies compared FC latanoprost/timolol versus latanoprost and timolol administered concomitantly 3536.

Four studies compared travoprost and latanoprost 12132337. Two studies compared travoprost and timolol 2338. Two studies compared travoprost or FC-containing travoprost versus latanoprost (or FC-containing latanoprost) 3940 and three studies compared travoprost or FC-containing travoprost versus timolol (or FC-containing timolol) 394041. Finally, 16 publications contained other comparisons of treatment types that were not duplicated with any other included study. 42434445464748495051525354555657

Characteristics of trials

Table 2 shows the characteristics of the trials as well as the quality of reporting. The average sample size was 311 (SD 33), although on average, only 86% of patients were included in the final analyses. Seventeen out of the included 47 trials (36%, 95% CI: 24–51) were crossover designs.

Table 2

Methodological quality and reporting

Characteristics

Number of studies with characteristics

% of studies with characteristics

95% Confidence Interval

Crossover studies

24, 51

Parallel treatment studies

49, 76

Title and Abstract specifying that trial is a noninferiority or equivalence trial

6, 25

Articles whose background or introduction section contains a rationale for using a noninferiority or equivalence trial design

5, 23

Articles with a clear objective or hypothesis pertaining to noninferiority or equivalence

17, 42

Sample size determination: Studies that appropriately described how the sample size was determined*

58, 83

Randomization: Studies stating the methods used in the generation of the random allocation sequence

33, 61

Masking:

Two or more groups

60, 85

Patients only

3, 20

Open label studies (i.e. studies that were not masked)

9, 30

Allocation concealment: Studies reporting this characteristic

12, 35

Statistical methods and numbers analyzed:

Studies that employed the combined test of noninferiority and superiority

9, 30

Studies presenting results of only ITT population with no mention of PP population

24, 51

Studies presenting results of only PP population with no mention of ITT population

17, 42

Studies presenting only results of ITT population but mentioned that PP produced similar results

15, 39

Studies presenting only results of PP population but mentioned that ITT produced similar results

1, 14

Studies reporting and presenting results of both ITT and PP populations

2, 17

Results:

Studies presenting a figure showing the range of the confidence interval relative to the pre-specified noninferiority/equivalence margin

0.5, 11

* = appropriate sample size determination stating all of the following; a predetermined noninferiority or equivalence margin, standard deviation used, significance level [α], and the power used

Of the 47 trials, 36 were noninferiority trials (77%, 95% CI: 63–86) and 11 (23%, 95% CI: 14–37) were designed for equivalence. In the final interpretations of the studies, 16/47 (34%, 95% CI: 22–48) trials claimed noninferiority, 14/47 (30%, 95% CI: 19–44) claimed equivalence and 17/47 (36%, 95% CI: 24–51) claimed superiority. For the trials claiming noninferiority, 33/36 (92%, 95% CI: 78–92) were accurate within their noninferiority margin. For those claiming equivalency, 10/11 (90%, 95% CI: 62–98) were accurate within their a priori margins. Seventeen percent of trials (8/47, 95% CI: 9–30) employed a combined test of noninferiority and superiority.

Sequence generation was reported in 22/47 trials (47%, 95% CI: 33–61). Allocation concealment was reported in only 10/47 (21%, 95% CI: 12–35) of the trials. Thirty-five studies (74%, 95% CI: 60–85) employed masking of at least two groups, 4/47 (9%, 95% CI: 3–20) masked only patients and 8/47 (17%, 95% CI: 9–30) were open label studies.

Only 6 of the 47 (13%, 95% CI: 6–25) studies specified that the trial was a noninferiority or equivalence trial in the title or abstract. The background or introduction section of 5/47 (11%, 95% CI: 5–23) articles contained a rationale for using a noninferiority or equivalence trial design. Thirteen articles (28%, 95% CI: 17–42) had a clear objective or hypothesis pertaining to noninferiority or equivalence. Thirty-four trials (72%, 95% CI: 58–83) properly described the method applied in the sample size determination and set appropriate boundaries on noninferiority or equivalence. The pre-stated noninferiority/equivalence in all the trials lied between the ranges of -1 to 2.5. Primary outcome measure was the mean intraocular pressure which was mostly measured as a mean difference or as a mean reduction from baseline.

Only 3/47 (6%, 95% CI: 2–17) studies reported and presented results of both ITT and PP populations. Two studies (4%, 95% CI: 1–14) presented only PP results but mentioned that ITT population had similar results. Twelve studies (26%, 95% CI: 15–39) presented only ITT results but mentioned that PP population had similar results. Thirteen trials (28%, 95% CI: 17–42) presented only PP results with no mention of ITT population results while 17/47 (36%, 95% CI: 24–51) studies presented only ITT results with no mention of PP population results.

From the 47 trials reported in this study, only one (2%, 95% CI: 0.5–11) diagrammatically presented its results by use of a figure showing the confidence intervals and pre-specified margins of equivalence or noninferiority. Handling of losses to follow-up was not addressed in 33 of the 47 trials (70%, 95% CI: 56–81). Loss to follow-up was mentioned but unaddressed in 10 trials (21%, 95% CI: 12–35), while 3/47 (6%, 95% CI: 2–17) were addressed in the statistics and 1/47 (2%, 95% CI: 0–11) had a zero loss to follow-up.

Measurement of hyperemia

We found large heterogeneity of measuring hyperemia. Two of the 47 (4%, 95% CI: 1–14) included studies used a severity scale based on non-serious, mild, moderate, and serious. Six studies (13%, 95% CI: 6–25) used a severity scale based on none, mild, moderate, and serious. Five (11%, 95% CI: 5–23) used a scale that recorded only mild, moderate, and severe. One (2%, 95% CI: 0.5–11) used a scale that included only non-serious and serious. One (2%, 95% CI: 0.5–11) used a four-point scale from 1–4 stating minimum and maximum as the scale endpoints. Six (13%, 95% CI:6–25) studies used a 5 point scale with 0 = none, 0.5 = trace, 1 = mild, 2 = moderate, and 3 severe. A further 26/47 (55%, 95% CI: 41–69) did not report their method of measuring hyperemia, of which 5/26 (19%, 95% CI: 9–38) did not report on hyperemia outcomes.

Discussion

To our knowledge, this is the first study that has attempted to look into the methodological quality and reporting of any ophthalmology RCTs designed to show noninferiority or equivalence for glaucoma treatments. These findings reinforce the gaps that exist in the methodological quality and reporting of these trials in general. We additionally found that within ophthalmology trials, a substantial heterogeneity exists in terms of measuring clinical outcomes, such as hyperemia. It is clear that further efforts at improving the reporting and analyses of clinical trials in this field are needed. Recent work by Hopewell and colleagues 58 highlighted the need for journals to endorse the CONSORT Statement and its extensions, and also make its application a condition for authors wishing to publish in the journal. Journals in the ophthalmology field should consider such an endorsement as well as promoting a strict adherence to the extended CONSORT statement in order to potentially improve manuscript reporting.

There are several strengths and limitations to consider in interpreting our study. Strengths include our extensive searching of the literature to identify a representative sample of a field that commonly utilizes noninferiority and equivalence trials. Our data abstraction methods aimed to reduce investigator driven bias with duplicate data abstraction. Limitations to consider include that we solely relied upon the reporting of specific items rather than inquiring about any missing items through contact with each individual trial investigator. As outlined in the methods, study authors were only contacted when we were unclear whether the trial itself included a non-inferiority or equivalence study design. It is possible that some of the specific methodological items were conducted but not reported. Indeed, this issue has been highlighted by investigators 59. However, our study aimed to determine reporting quality in order to make specific recommendations on transparent reporting. We did not determine changes pre- versus post-publication of the extended CONSORT statement in 2006. Previous research has found that the quality of reports improved after publication of the original 1996 CONSORT publication 60. Per-protocol analyses have traditionally been preferred over ITT approaches despite current consensus of displaying both 147. In our study we were unable to determine if one is superior to the other as we found too few examples where both outcomes are reported.

The findings of our study are consistent with other studies investigating reporting quality in general medicine journals and specialist journals 61. General medicine journals tend to report important methodological recommendations more frequently than specialist journals. This may be due to the encouragement of journals to utilize the CONSORT recommendations through editorials and letters 62. Providing editorials or guidelines in the instructions to authors is a potential option for improving the reporting of these RCTs. An option that may be considered is a checklist submitted with clinical trials indicating where in the manuscript specific details are reported, as required by many current journals.

An important methodological issue that has been raised by this review is the use of a combined test of noninferiority and superiority. The purpose of conducting noninferiority and equivalence trials is not to evaluate superiority, as that would require far more power than would be expected in this type of trial and would maintain the need for ITT analysis 63. In our study, 8 trials employed the combined test for noninferiority and superiority. This allows a trial to claim its original intent of noninferiority if superiority is not identified, but can claim superiority if the arbitrary threshold, upper-limit of noninferiority, is surpassed 64.

Given the heated market of the prostaglandin field, we found that several (n = 17) noninferiority and equivalence trials claimed superiority of the test intervention over competitors products, regardless of overall findings and the study intent. For example, in a randomized trial evaluating travoprost to latanoprost on 408 patients' daily intraocular pressure at 9 am, 11 am and 4 pm, the authors found noninferiority, but report on statistical superiority at the 9 am measurement 40. Given the lack of ITT analysis, the multiple testing and initial intention of a noninferiority analysis, such findings provide misleading evaluations of superiority. Indeed, this phenomenon exists across a range of medical fields 656667686970.

Pre-specifying the noninferiority or equivalence margin is necessary to provide strong inferences about the suitability of one treatment compared to another. If the margin employed is too wide, risk for type 1 error exists. Similarly, although unlikely, if one were to establish the margin too narrow, a type 2 error is possible. Margins should be established based on clinical justification and not statistical rule 71.

Conclusion

In conclusion, based on the analysis of the study reports that we have examined, our findings demonstrate deficiencies in the design, planning, and reporting of noninferiority and equivalence trials in the ophthalmology literature. Many studies of clinical noninferiority and equivalence do not set boundaries for equivalence. Claims of "superiority" and "similarity" are often displayed despite the use of improper analysis. These methodologic deficiencies appear to lead to false claims, unrepeatable experiments, and possible costs or harms to patients.

Competing interests

Over the past 5 years, Edward Mills has consulted to Pfizer Ltd. Edward Mills is the recipient of an industry partnered New Investigator award from The Canadian Institutes of Health Research to conduct methodological research. Chia-Wen Lee has been an employee of Pfizer Ltd. No other authors have any competing interests. We can think of no way that the declared competing interests can benefit from this manuscript.

Authors' contributions

OE, BR, CWL, and EM conceptualized the study. OE, BR, CWL, and EM acquired the data. OE, BR, CWL, and EM analyzed the data. OE, BR, CWL, and EM wrote the manuscript. OE, BR, CWL, and EM approved the final manuscript.

Acknowledgements

Funding for this study was provided by Pfizer Ltd in the form of an educational grant.

Reporting of noninferiority and equivalence randomized trials Piaggio G Elbourne DR Altman DG Pocock SJ Evans SJW JAMA 2006 295 1152 1160 10.1001/jama.295.10.1152 16522836 Guidance for Industry, E 10, Choice of control group and related issues in clinical trials United States Food and Drug Administration (FDA) 2001 http://www.fda.gov/cder/guidance/4155fnl.pdf Lessons from and cautions about noninferiority and equivalence randomized trials Gøtzsche PC JAMA 2006 295 1172 1174 10.1001/jama.295.10.1172 16522840 Quality of reporting of noninferiority and equivalence randomized trials Le Henanff A Giraudeau B Baron G Ravaud P JAMA 2006 295 1147 1151 10.1001/jama.295.10.1147 16522835 A comparison of intent-to-treat and per-protocol results in antibiotic non-inferiority trials Brittain E Lin D Statist Med 2005 24 1 10 10.1002/sim.1934 Statistical principles for clinical trials The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) 1998 E9 http://www.ich.org/LOB/media/MEDIA485.pdf Points to consider on switching between superiority and non-inferiority Committee for Proprietary Medicinal Products (CPMP) 2000 http://www.emea.europa.eu/pdfs/human/ewp/048299en.pdf Practical issues in equivalence trials Ebbutt AF Frith L Statist Med 1998 17 1691 1701 10.1002/(SICI)1097-0258(19980815/30)17:15/16<1691::AID-SIM971>3.0.CO;2-J Agreement between alternative classifications of acute respiratory distress syndrome Meade MO Guyatt GH Cook RJ Groll R Kachura JR Wigg M Cook DJ Stutsky AS Stewart TE Am J Respir Crit Care Med 2001 163 490 493 11179128 Confidence intervals vs. Bayesian intervals Jaynes ET Foundations of Probability Theory, Statistical Inference, and Statistical Theories of Science Dordrecht: D. Reidel Harper WL, Hooker CA 1976 Three-month comparison of bimatoprost and latanoprost in patients with glaucoma and ocular hypertension Gandolfi S Simmons ST Sturm R Chen K Van Denburgh AM for the Bimatoprost Study Group Adv Ther 2001 18 110 121 10.1007/BF02850299 11571823 A comparison of latanoprost, bimatroprost, and travoprost in patients with elevated intraocular pressure: a 12-week, randomized, masked-evaluator multicenter study Parrish RK Palmberg P Sheu WP for the XLT Study Group Am J Ophthalmol 2003 135 688 703 10.1016/S0002-9394(03)00098-9 12719078 Intraocular pressure, safety, and quality of life in glaucoma patients switching to latanoprost from monotherapy treatments Zimmerman T Stewart WC for the Latanoprost Axis Study Group J Ocul Pharmacol Ther 2003 19 405 15 10.1089/108076803322472971 14583133 A six-month randomized clinical trial comparing the intraocular pressure-lowering efficacy of bimatoprost and latanoprost in patients with ocular hypertension or glaucoma Noecker RS Dirks MS Choplin NT Berstein P Batoosingh AL Whitcup SM for the Bimatoprost/Latanoprost Study Group Am J Ophthalmol 2003 135 55 63 10.1016/S0002-9394(02)01827-5 12504698 Efficacy and safety of bimatoprost in patients with elevated intraocular pressure: a 30-day comparison with latanoprost DuBiner H Cooke D Dirks M Stewart WC Van Denburgh AM Felix C Surv Ophthalmol 2001 44 S353 360 10.1016/S0039-6257(01)00212-0 A randomized, double-masked, multicentre clinical trial comparing bimatoprost and timolol for the treatment of glaucoma and ocular hypertension Whitcup SM Cantor LB Van Denburgh AM Chen K Br J Ophthalmol 2003 87 57 62 1771463 12488264 10.1136/bjo.87.1.57 Two-year double-masked comparison of bimatoprost with timolol in patients with glaucoma or ocular hypertension Cohen JS Gross RL Cheetham J Van Denburgh AM Bernstein P Whitcup SM Surv Opthalmol 2004 49 S45 52 10.1016/j.survophthal.2003.12.019 One-year randomized study comparing bimatoprost and timolol in glaucoma and ocular hypertension Higginbotham EJ Schuman JS Goldberg I Gross RL Van Denburgh AM Chen K Whitcup SM for the Bimatoprost Study Groups 1 and 2 Arch Ophthalmol 2002 120 1286 1293 12365906 Comparison of once- or twice-daily bimatoprost with twice-daily timolol in patients with elevated IOP: a 3-month clinical trial Brandt JD Van Denburgh AM Chen K Whitcup SM for the Bimatoprost Study Group Ophthalmology 2001 108 6 1023 1031 10.1016/S0161-6420(01)00584-X 11382623 Prospective comparative switch study from timolol 0.5% and latanoprost 0.005% to bimatoprost 0.03% Brittain CJ Saxena R Waldock A Adv Ther 2006 23 68 73 10.1007/BF02850348 16644608 Safety and efficacy of bimatoprost 0.03% versus timolol maleate 0.5%/dorzolamide 2% fixed combination Day DG Sharpe ED Beischel CJ Jenkins JN Stewart JA Stewart WC Eur J Ophthalmol 2005 15 336 342 15945001 A 3-month randomized controlled trial of bimatoprost (LUMIGAN) versus combined timolol and dorzolamide (COSOPT) in patients with glaucoma or ocular hypertension Coleman AL Lerner F Bernstein P Whitcup SM Ophthalmol 2003 110 2362 2368 Travoprost compared with latanoprost and timolol in patients with open-angle glaucoma or ocular hypertension Netland PA Landry T Sullivan EK Andrew R Silver L Weiner A Mallick S Dickerson J Bergamini MV Robertson SM Davis SS for the Travoprost Study Group Am J Ophthalmol 2001 132 472 484 10.1016/S0002-9394(01)01177-1 11589866 Intraocular pressure of 24 hours after repeated administration of latanoprost 0.0055 or timolol gel-forming solution 0.5% in patients with ocular hypertension Larsson LI Ophthalmology 2001 108 8 1439 1444 10.1016/S0161-6420(01)00605-4 11470697 A three-year prospective, randomized and open comparison between latanoprost and timolol in Japanese normal-tension glaucoma patients Tomita G Araie M Ktazawa Y Tsukhara S Eye 2004 18 984 989 10.1038/sj.eye.6701373 15037889 A six-month, randomized, double-masked study comparing latanoprost with timolol in open-angle glaucoma and ocular hypertension Watson P Stjernschantz J Ophthalmology 1996 103 1 126 137 8628543 Comparison of 24-hour intraocular pressure reduction with two dosing regimens of latanoprost and timolol maleate in patients with primary open-angle glaucoma Konstas AG Maltezos AC Gandi S Hudgins AC Stewart WC Am J Ophthalmol 1999 128 15 20 10.1016/S0002-9394(99)00073-2 10482089 Comparison between latanoprost and brimonidine efficacy and safety in Indian eyes Thomas R Parikh R Muliyil J George R Paul P Abraham LM Indian J Ophthalmol 2003 51 123 128 12831141 24-hour intraocular pressures with brmonidine purite versus dorzomalide added to latanoprost in primary open-angle glaucoma subjects Konstas AG Karabatsas CH Lallos N Georgiadis N Kotsimpou A Stewart JA Stewart WC Ophthalmology 2005 112 4 603 608 10.1016/j.ophtha.2004.11.032 15808251 Three-month comparison of brimonidine and latanoprost as adjunctive therapy in glaucoma and ocular hypertension patients uncontrolled on beta-blockers; tolerance and peak intraocular pressure lowering Simmons ST Earl ML for the Alphagan/Xalatan Study Group Ophthalmology 2002 109 2 307 314 10.1016/S0161-6420(01)00936-8 11825814 Efficacy and safety of the fixed combinations latanoprost/timolol versus drozolamide/timolol in patients with elevated intraocular pressure Shin DH Feldman RM Sheu WP for the Fixed Combination Latanoprost/Timolol Study Group Ophthalmology 2004 111 2 276 282 10.1016/j.ophtha.2003.05.019 15019375 Daytime diurnal curve comparison between the fixed combinations of latanoprost 0.005%/timolol maleate 0.5% and dorzomalide 2%/timolol maleate 0.5% Konstas AG Kozobolis VP Lallos N Christodoulakis E Stewart JA Stewart WC Eye 2004 18 1264 1269 10.1038/sj.eye.6701446 15218522 A comparison of the fixed combination of latanoprost <it>and timolol with its individual components</it> Pfeiffer N for the European Latanoprost Fixed Combination Study Group Graefes Arch Clin Exp Ophthalmol 2002 240 893 899 10.1007/s00417-002-0553-0 12486510 Latanaprost and timolol combination therapy vs. monotherapy: one year randomized trial Higginbotham EJ Feldman RJ Stiles M Dubiner H for the Fixed Combination Investigative Group Arch Ophthalmol 2002 120 915 922 12096962 A 12-week, randomized, double-masked, multicenter study of the fixed combination of latanoprost and timolol in the evening versus individual components Diestelhorst M Larsson LI for the European-Canadian Latanoprost Fixed Combination Study Group Ophthalmology 2006 113 1 70 76 10.1016/j.ophtha.2005.06.027 16263174 A 12 week study comparing the fixed combination of latanprost and timolol with the concomitant use of the individual components in patients with open angle glaucoma and ocular hypertension Diestelhorst M Larrson LI for the European Latanoprost Fixed Combination Study Group Br J Ophthalmol 2004 88 199 203 1771993 14736774 10.1136/bjo.2003.018234 Efficacy and safety of latanoprost versus travoprost in exfoliative glaucoma patients Konstas AG Kozobolis VP Katsimpris IE Boboridis K Koukoula S Jenkins JN Stewart WC Ophthalmology 2007 114 4 653 657 10.1016/j.ophtha.2006.07.064 17197028 Comparison of topical travoprost eye drops given once daily and timolol 0.5% given twice daily in patients with open-angle glaucoma or ocular hypertension Goldberg I Cunha-Vaz J Jakobsen JE Nordmann JP Trost E Sullivan EK for the International Travoprost Study Group J Glaucoma 2001 10 414 422 10.1097/00061198-200110000-00010 11711841 A 6-week, double-masked, parallel-group study of the efficacy and safety of travoprost 0.004% compared with latanoprost 0.005%/timolol 0.5% in patients with primary open-angle glaucoma or ocular hypertension Franks WA Renard JP Cunlife IA Rojanapongpun P Clin Ther 2006 28 332 339 10.1016/j.clinthera.2006.03.001 16750448 A 1-year study to compare the efficacy and safety of once-daily travoprost 0.004%/timolol 0.5% to once-daily latanoprost 0.005%/timolol 0.5% in patients with open-angle glaucoma or ocular hypertension Topouzis F Melamed S Danesh-Meyer H Wells AP Kozobolis V Wieland H Andrew R Wells D Eur J Ophthalmol 2007 17 183 190 17415690 A three-month, multicenter, double-masked study of the safety and efficacy of travoprost 0.004%/timolol 0.5% ophthalmic solution compared to travoprost 0.004% ophthalmic solution and timolol 0.5% dosed concomitantly in subjects with open-angle glaucoma or ocular hypertension Hughes BA Bacharach J Craven ER Kaback MB Mallick S Landry TA Bergamini MV J Glaucoma 2005 14 392 299 10.1097/01.ijg.0000176935.08392.14 16148589 Efficacy and tolerability of the dorzolamide 2%/timolol 0.5% combination (COSOPT) versus 0.005% (XALATAN) in the treatment of ocular hypertension or glaucoma: results from two randomized clinical trials Fechtner RD Airaksinen PJ Getson AJ Lines CR Adamsons IA for the COSOPT versus XALATAN Study Groups Acta Ophthalmol Scand 2004 82 42 48 10.1046/j.1600-0420.2004.0205.x 14982045 A double-masked randomized parallel comparison of a fixed combination of bimatoprost 0.03%/timolol 0.5% with non-fixed Hommer A Eur J Ophthalmol 2007 17 53 62 17294383 The efficacy and safety of timolol maleate versus brinzolamide each given twice daily added to travoprost in patients with ocular hypertension or primary open-angle glaucoma Holló G Chiselita D Petkova N Cvenkel B Liehneova I Izgi B Berta A Szaflik J Turacli E Stewart WC Eur J Ophthalmol 2006 16 816 823 17191187 Brimonidine purite and bimatoprost compared with timolol and latanoprost in patients with glaucoma and ocular hypertension Netland PA Mroz M Rosner SA Katzman B Macy JI Adv Ther 2003 20 20 30 10.1007/BF02850116 12772815 Efficacy and safety of the latanoprost/timolol maleate fixed combination vs concomitant brimonidine and latanoprost therapy Stewart WC Stewart JA Day DG Sharpe ED Jenkins JN Eye 2004 18 990 995 10.1038/sj.eye.6701375 15037890 Timolol 0.5%/dorzolamide 2% fixed combination vs timolol maleate 0.5% and unoprostone 0.15% given twice daily to patients with primary open-angle glaucoma or ocular hypertension Day DG Schacknow PN Wand M Sharpe ED Stewart JA Leech J Stewart WC Am J Ophthalmol 2003 135 138 143 10.1016/S0002-9394(02)01887-1 12566015 Brimonidine 0.2% vs unoprostone 0.15% both added to timolol maleate 0.5% given twice daily to patients with primary open-angle glaucoma or ocular hypertension Sharpe ED Henry CJ Mundorf TK Day DG Stewart JA Jenkins JN Stewart WC Eye 2005 19 35 40 10.1038/sj.eye.6701392 15094733 Comparison of the 24-hour intraocular pressure-lowering effects of latanorpost and dorzolamide/timolol fixed combination after 2 and 6 months of treatment Konstas AGP Kozobolis VP Tsironi S Makridaki I Efremova R Stewart WC Ophthalmology 2008 115 1 99 103 10.1016/j.ophtha.2007.03.007 18166407 Efficacy and safety of a fixed combination of travoprost 0.004%/timolol 0.5% ophthalmic solution once daily for open-angle glaucoma or ocular hypertension Schuman JS Katz GJ Lewis RA Henry JC Mallick S Wells DT Sullivan EK Landry TA Bergamini MVW Robertson SM Am J Ophthalmol 2005 140 242 250 16086946 Efficacy and safety of timolol maleate/latanoprost fixed combination versus timolol maleate and brimonidine given twice daily Stewart WC Stewart JA Day D Sharpe ED Acta Ophthalmol Scand 2003 81 242 246 10.1034/j.1600-0420.2003.00065.x 12780402 A comparison of the effects on intraocular pressure of latanoprost 0.005% and the fixed combination of dorzolamide 2% and timolol 0.5% in patients with open-angle glaucoma Honrubia FM Larsson LI Spiegel D the European Latanoprost Study Group Acta Ophthalmol Scand 2002 80 635 641 10.1034/j.1600-0420.2002.800615.x 12485286 A double-masked, randomized clinical trial comparing latanoprost with unoprostone in patients with open-angle glaucoma or ocular hypertension Susanna R Jr Giampani J Borges AS Vessani RM Jordao MLS Ophthalmol 2001 108 259 263 The additive intraocular pressure-lowering effect of latanoprost 0.005% daily once and pilocarpine 2% t.i.d. in patients with open-angle glaucoma or ocular hypertension. A 6-month, randomized, multicenter study Diestelhorst M for the German Latanoprost Study Group Graef Arch Clin Exp Ophthalmol 2000 238 433 439 10.1007/s004170050375 Efficacy and safety of timolol solution once daily vs timolol gel added to latanoprost Stewart WC Day DG Sharpe ED Dubiner HB Holmes KT Stewart JA Am J Ophthalmol 1999 128 692 696 10.1016/S0002-9394(99)00237-8 10612504 Comparison of the additive intraocular pressure-lowering effect of latanoprost and dorzolamide when added to timolol in patients with open-angle glaucoma or ocular hypertension: a randomized, open-label, multicenter study in Greece Petounis A Mylopoulos N Kandarakis A Andreanos D Dimitrakoulias N J Glaucoma 2001 10 316 324 10.1097/00061198-200108000-00012 11558817 A comparison of once-daily morning vs evening dosing of concomitant latanoprost/timolol Konstas AGP Nakos E Tersis I Lallos NA Leech JN Stewart WC Am J Ophthalmol 2002 133 753 757 10.1016/S0002-9394(02)01460-5 12036665 Endorsement of the CONSORT Statement by high impact factor medical journals: a survey of journal editors and journal 'Instructions to Authors' Hopewell S Altman DG Moher D Schulz KF Trials 2008 9 20 2359733 18423021 10.1186/1745-6215-9-20 An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods Devereaux PJ Choi PT El-Dika S Bhandari M Montori VM Schünemann HJ Garg AX Busse JW Heels-Ansdell D Ghali WA Manns BJ Guyatt GH J Clin Epidemiol 2004 57 1232 1236 10.1016/j.jclinepi.2004.03.017 15617948 Does the CONSORT checklist improve the quality of reports of randomized controlled trials: a systematic review Plint AC Moher D Morrison A Schulz KF Altman DG Hill C Gaboury I Medical Journal of Australia 2006 185 263 267 16948622 The role of intention-to-treat in analysis of noninferiority studies Wiens BL Zhao W Clin Trials 2007 4 286 291 10.1177/1740774507079443 17715258 An analysis of general medical and specialist journals that endorse CONSORT found that reporting was not enforced consistently Mills E Wu P Gagnier J Heels-Ansdell D Montori VM J Clin Epidemol 2005 58 662 667 10.1016/j.jclinepi.2005.01.004 Lies, damn lies, and reports of RCTs McNamee D Horton R Lancet 1996 348 562 10.1016/S0140-6736(05)64795-2 8774566 The prostaglandin wars Kaufman R Am J Ophthalmol 2003 136 727 728 10.1016/S0002-9394(03)00658-5 14516814 Randomized clinical trial comparing intraocular pressure-lowering efficacy of bimatoprost and latanoprost in patients with ocular hypertension or glaucoma Eisenberg D Am J Ophthalmol 2003 136 217 10.1016/S0002-9394(03)00195-8 12834708 Are claims of equivalency in digestive diseases trials supported by the evidence? Tinmouth JM Steele LS Tomlinson G Glazier RH Gastroenterol 2004 126 1700 1710 10.1053/j.gastro.2004.03.005 Claims of equivalence in randomized controlled trials of the treatment of bacterial meningitis in children Krysan DJ Kemper AR Pediatr Infect Dis J 2002 21 753 758 10.1097/00006454-200208000-00011 12192164 Claims of equivalence in medical research: are they supported by the evidence? Greene WL Concato J Feinstein AR Ann Intern Med 2000 132 715 722 10787365 Active-control equivalence trials and antihypertensive agents McAlister FA Sackett DL Am J Med 2001 111 553 558 10.1016/S0002-9343(01)00900-7 11705432 Methodological standards in non-inferiority AIDS trials: moving from adherence to compliance Parienti JJ Verdon R Massari V BMC Med Res Methodol 2006 6 46 1592102 16987409 10.1186/1471-2288-6-46 Choosing an equivalence limit for non-inferiority or equivalence studies Weins BL Control Clin Trials 2002 23 2 14 10.1016/S0197-2456(01)00196-9 11852160