Trials - Latest Comments

Efficacy due to alcohol or something else in the mouthwash composition?

Dirk Lachenmeier — 2012-03-27T10:36:03Z

As it was previously noted, a considerable industry bias in mouthwash-related research exists [1]. Therefore, it would be interesting to know the sponsor of the study and if the sponsor was affiliated with any of the mouthwash brands under investigation.
To provide insight into the transferability of the results, data about the composition of the two mouthwash types under investigation would be important as the composition is highly variable between brands and types [2,3]. Did the two products have the same content of essential oils? What kinds of essential oils, which may have different efficacy dependent on type, were used? At least the manufacturer of the alcohol-containing products provides different product types with different composition, which one was it? Would it not have been a better experimental design to use the same formulation with and without alcohol than to use two commercial types of mouthwash, which may have additional differences other than the alcohol content? How can the conclusion that the difference is due to alcohol uphold, when the composition has not been characterized at all?

[1] Lachenmeier DW: Safety evaluation of topical applications of ethanol on the skin and inside the oral cavity. J Occup Med Toxicol 2008, 3:26. http://www.occup-med.com/content/3/1/26
[2] Lachenmeier DW, Keck-Wilhelm A, Sauermann A, Mildau G: Safety assessment of alcohol-containing mouthwashes and oral rinses. SOFW J 2008, 134:70-78.
[3] Lachenmeier DW, Gumbel-Mako S, Sohnius EM, Keck-Wilhelm A, Kratz E, Mildau G: Salivary acetaldehyde increase due to alcohol-containing mouthwash use: a risk factor for oral cancer. Int J Cancer 2009, 125:730-735.

Comparing apples and oranges

Joselita Salita — 2011-07-21T12:45:16Z

I appreciate the goals of Nairy et al.'s paper. However, their primary objective was incomplete which made them design an experiment which compared apples and oranges. The three groups could only be partially compared: group 1 vs 2 and 1 vs 3, which in the end, could not support their conclusions.

In order to assess the efficacy of in-situ gels in treating OPC, one compares in-situ gels with a placebo or a comparator (in this case, flucanozole tablets) or both among one patient type. Having two types of patients, with one type incompletely represented was a major flaw. The choice of comparing responses among HIV/AIDS patients as a patient type was also not a good idea due to confounding effects. The resulting „good clinical response“ of HIV patients after 21 days in which 85% in Group I receiving in situ gels were considered OPC cured compared to 66% HIV patients in Group III treated with tablets could be insignificant not only due to small sample size (n=15) but also to confounding effects. So did relapse of 80 and 90% respectively. Group II was comprised of non-HIV patients and was treated with in situ gels. Better outcomes for this group (clinical response=100% and relapse=0) may have resulted because the subjects were HIV-free (therefore responded better) and similar results could have been obtained if this group received placebo in-situ gel or flucanozole tablets. For this particular data set no statistical tests were performed but comparison was not valid anyway.

All other data with or without statistical support showed the same trends, that is, Group II having the most positive outcome and Group III the least. I think they have no relevance under the same reasoning. The only valid conclusion in this paper is that in situ gels are more efficacious in patients without HIV/AIDS than those with HIV/AIDS. Whether in situ gels are better than tablets among healthy or HIV patients is unanswered.

To the Editor

Dirk Kuhlmann — 2011-06-23T12:09:45Z

The study showing an in situ gel formulation is an interesting approach for treatment of OPC. The clinical study data presented by Nairy et al. is poorly organized and confusing.

The reliability of the results is questionable due to incomplete outcome data and selection bias from randomization based on clinical evaluation. The three study groups are only partially comparable. As group II is missing a control group, the data is not supportive of the aim of the study.

Severity of signs and symptoms are mild to absent according to table 3. An effect of treatment on moderate to severe cases is not presented. Patient demographics of group 3 are diverse with a possible confounding bias associated with a large variance in CD4 count.

The primary endpoints are not specified to be at EOT or after follow-up. Data and results presented throughout the paper are not consistent thereby decreases their validity. Gender counts for group II vary in the study design and table 2. Changes in symptoms vary throughout the paper. In the abstract, 97% of group I were cured. In the discussion, 95% of this group were cured but only 80% showed a good clinical response in the in vivo evaluation up to 21 days. At EOT, group III showed 85% improvement in the abstract and 93% and 66.6% as cured in the discussion section and in vivo evaluation respectively. The results in figure 2 are not supportive of other data. Percentage points do not correlate to natural numbers of patients.

In group I and II, 40% of the patients experienced gastrointestinal symptoms. Unfortunately no mention of adverse events is made for the control group.

According to the authors, there is no significant difference in clinical response at EOT and at end of follow-up and no improvement in group I and III at the end of the study.
In my opinion, the data is not supportive for concluding a better clinical efficacy in HIV/AIDS patients. Successful clinical outcome was only reported during the treatment duration. Insufficient information is given to discern whether the in situ gel is a viable alternative to conventional formulations.

It would be advantageous if the authors could repeat the study in a larger, randomized patient group with a suitable control group and study setup and with treatment endpoints comparable to other OPC studies.

Yours sincerely
D. Kuhlmann

Systematizing the analysis of effect heterogeneity requires rethinking some fundamentals

James Scanlan — 2011-06-01T10:30:15Z

The article by Gabler et al.[1] questions the soundness of epidemiological literature’s reporting and analysis of heterogeneity of treatment effects (HTE) and calls for greater attention to HTE issues and more systematic analysis of such issues.

But the goals the authors seek cannot be achieved without reconsideration of certain fundamentals of subgroup analysis. Standard approaches to such analyses are based on an assumption that absent HTE all subgroups will experience equal proportionate changes in outcome rates (i.e., the same rate ratio across different baseline rates) and that HTE is observed in those cases where equal proportionate changes are not found. That assumption, however, is demonstrably unsound for the simple reason that it is not possible for a factor to cause equal proportionate changes in an outcome while causing equal proportionate changes in the opposite outcome.

That is, for example, if Group A has a baseline rate of 5% and Group B has a baseline rate of 10%, a factor that reduces the two rates by equal proportionate amounts, say 20% (from 5% to 4% and from 10% to 8%) would necessarily increase the opposite outcome by different proportionate amounts (95% increased to 96%, a 1.05% increase; 90% to 92%, a 2.2% increase). And since there is no more reason to expect that two group would undergo equal proportionate changes in one outcome than there is to expect they would undergo equal proportionate changes in the opposite outcome, there is no reason to expect that the two groups would undergo equal proportionate changes in either outcome.

For reasons inherent in the shapes of normal distributions of factors associated with experiencing or avoiding an outcome, it is more reasonable to expect that a treatment that reduces an outcome rate will tend to cause a larger proportionate decrease in that outcome for groups with the lower base rates while causing a larger proportionate increase in the opposite outcome for other groups.[2-6]

These considerations raise several issues for subgroup analyses. The first involves identifying benchmarks across baseline rates the departure from which would be deemed a subgroup effect. See Table 6 of the presentation in reference 4 for the application of the approach to identifying subgroup effects discussed in reference 4 to data in reference 49 of Gabler et al. A second, and more important, involves using information derived from observed treatment effects for particular groups to estimate absolute risk changes in groups for which treatment effect information is not available or not reliable.[5,6]

References:

1. Gabler NB, Naihua D, Liao D, et al. Dealing with heterogeneity treatments: is the literature up to the challenge. Trials 2009,10:43: http://www.trialsjournal.com/content/10/1/43 (Accessed May 29, 2011.)

2. Scanlan JP. Race and mortality. Society 2000;37(2):19-35: http://www.jpscanlan.com/images/Race_and_Mortality.pdf (Accessed May 29, 2011.)

3. Scanlan JP. Divining difference. Chance 1994;7(4):38-9,48: http://jpscanlan.com/images/Divining_Difference.pdf (Accessed May 29, 2011.)

4. Scanlan JP. Interpreting Differential Effects in Light of Fundamental Statistical Tendencies, presented at 2009 Joint Statistical Meetings of the American Statistical Association, International Biometric Society, Institute for Mathematical Statistics, and Canadian Statistical Society, Washington, DC, Aug. 1-6, 2009: PowerPointPresentation: http://www.jpscanlan.com/images/Scanlan_JSM_2009.ppt; Oral Presentation: http://www.jpscanlan.com/images/JSM_2009_ORAL.pdf (Accessed May 29, 2011.)

5. Scanlan’s Rule page of jpscanlan.com: http://jpscanlan.com/scanlansrule.html (Accessed May 29, 2011.)

6. Scanlan JP. Assessing heterogeneity of treatment effects in light fundamental statistical tendencies. Trials May 26, 2011(responding to Kent DM, Rothwell PM, Ionnadis JPA, et al. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials 2010,11:85): http://www.trialsjournal.com/content/11/1/85/comments#498686 (Accessed May 29, 2011.)

Assessing heterogeneity of treatment effects in light fundamental statistical tendencies

James Scanlan — 2011-05-26T15:41:35Z

The article by Kent et al.[1] provides useful guidance on the reporting of results by subgroup. But the article suffers from the common assumption that the absence of a subgroup effect (heterogeneity) is reflected by equivalent relative risk reductions across subgroups. It is not logical to regard equivalent relative risk reductions as somehow normal (i.e., as reflecting the absence of a subgroup effect) for the simple reason that it is not possible for a factor to cause equal relative reductions in an outcome for groups with different base rates while causing equal relative increases in the opposite outcome for those groups.

The point can be illustrated with figures in Table 1 of Kent et al., which shows the same 25% relative risk reduction of an adverse outcome for an average risk group (4% reduced to 3%) and a high risk group (20% reduced to 15%). Such figures would mean that the increase in the favorable outcome rate was 1% for the low risk group (96% increased to 97%) but 6.3% for the high risk group (80% increased to 85%). Since there is no more reason to regard it as somehow normal that there will be equal relative changes in one outcome than to regard it as somehow normal that there will be equal relative changes in the opposite outcome, there is no reason to regard it as somehow normal that there will be equal relative changes in either outcome.

In fact, for reasons related to the shapes of normal risk distributions there is reason to expect that a factor that reduces the risk of an outcome will tend to cause groups with lower base rates to experience larger relative reductions in those rates, while causing other groups to experience larger relative increases in the opposite outcome rates.[2-7] Thus, reliance on relative risk as a measure of effect size tends to yield opposite conclusions as to whether high or low risk groups derive the greater benefit from an intervention depending on whether one examines the favorable or the adverse outcome.

The only way to determine whether there exists a meaningful subgroup effect is to derive from the rates for controls and treated subjects in each subgroup the differences between means of the hypothesized underlying distributions.[2-4,7] Taking the data from Table 1 of Kent et al., the reduction from 4% to 3% for the average risk group reflects a differences between means of the underlying distributions of .13 standard deviations. An equivalent reduction in the risk for the high risk group would be reflected by a reduction from 20% to 17%. Determinations of meaningful interaction should be based on departures from patterns such as these, not departures from equivalent relative changes in either the adverse or the favorable outcome. Similarly, appraisals of the likely absolute risk reduction for each subgroup – as Kent et al. note, the key clinical consideration – should be based on benchmarks derived as just described rather than on benchmarks based on equivalent relative risk reductions.

References:

1. Kent DM, Rothwell PM, Ionnadis JPA, et al. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials 2010,11:85: http://www.trialsjournal.com/content/11/1/85 (Accessed May 1, 2011.)

2. Scanlan JP. Interpreting Differential Effects in Light of Fundamental Statistical Tendencies, presented at 2009 Joint Statistical Meetings of the American Statistical Association, International Biometric Society, Institute for Mathematical Statistics, and Canadian Statistical Society, Washington, DC, Aug. 1-6, 2009: http://www.jpscanlan.com/images/JSM_2009_ORAL.pdf;http://www.jpscanlan.com/images/Scanlan_JSM_2009.ppt (Accessed May 1, 2011.)

3. Scanlan JP. Rethinking the premises of subgroup analyses. BMJ June 7, 2010 (responding to Sun X, Briel M. Walter SD, and Guyatt GH. Is as subgroup effect believable? Updating criteria to evaluated the credibility of subgroup analyses. BMJ 2010;340:850-854): http://www.bmj.com/cgi/eletters/340/mar30_3/c117 (Accessed May 1, 2011.)

4. Scanlan JP. Problems in identifying interaction where groups have different base rates. BMJ Sept. 21, 2010 (responding to Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ 2003;326:219): Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ 2003;326:219): http://www.bmj.com/content/326/7382/219/reply#bmj_el_241943 (Accessed May 1, 2011.)

5. Scanlan JP. Race and mortality. Society 2000;37(2):19-35: http://www.jpscanlan.com/images/Race_and_Mortality.pdf (Accessed May 1, 2011.)

6. Scanlan JP. Divining difference. Chance 1994;7(4):38-9,48: http://jpscanlan.com/images/Divining_Difference.pdf (Accessed May 1, 2011.)

7. Subgroup Effects sub-page of Scanlan’s Rule page of jpscanlan.com: http://www.jpscanlan.com/scanlansrule/subgroupeffects.html (Accessed May 1, 2011.)

Erratum: minor error

Andrew Vickers — 2010-11-30T09:14:31Z

In the text of our protocol, we gave an incorrect affiliation to JianPing Liu. He is not, as we suggested, affiliated with the Chinese Cochrane Center, but is Director of the Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine. The last sentence of the "Search strategy for identification of studies" section should therefore read:

Chinese trials will be identified by a separate process: Jianping Liu of the Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, will use that institution's resources to identify Chinese trials of acupuncture for chronic pain that involved full allocation concealment.

Educational opportunities available to Trial and Data Managers

Rachel Breen — 2010-09-01T10:04:55Z

I read with interest the article (provisional pdf) regarding Trial Management.
Managing clinical trials
Trials 2010, 11:78 doi:10.1186/1745-6215-11-78
Barbara Farrell, Sara Kenyon, Haleema Shakur.

However I felt that the article did not provide a balanced review of the educational opportunities available to Trials Managers and Data Managers. The article only mentioned the London School of Hygiene and Tropical Medicine's MSc in Clinical Trials. There are other MSc courses that have been developed to address trial management needs and could be argued to be more specific to the needs of Trial Managers than LSHTM. Examples include the Cranfield MSc and also MSc Clinical Research Administration run by the University of Liverpool, which is conducted completely in an online learning environment and was developed to meet the needs of Trials Managers in particular. This MSc encourages students to undertake research dissertations relevant to the practices of trial management. Typical examples of dissertation projects include: core outcome sets (an agreed standardised minimum set of outcomes that should be measured and reported in all clinical trials of effectiveness in a specific condition), consent issues, data capture, ethics committees in multicentre/multinational studies, non-response bias, study recruitment, trial monitoring, adherence to trial reporting standards and issues of relevance to the pharmaceutical industry or specific clinical areas.
This MSc is also linked to the activities of the MRC NWHTMR (www.methodologyhubs.mrc.ac.uk) in part due to the relevance of the dissertation projects. I would also recommend that interested readers contact the MRC Hub network to pursue trial management research.

Searching for new insights in subgroup analyses

Ewout Steyerberg — 2009-11-24T12:50:11Z

The SATIRE group is to be applauded for their effort to shed more light at the minefield of subgroup analysis, reporting, and interpretation. The question may however be what truly new insights will be obtained. One might predict that findings from a well-selected set of papers published in 2007 will be largely similar to what was found before in the studies shown in Table 1.

Of particular interest may be the answer to question 10 in Appendix 1: "Is the interaction consistent across closed related outcomes within the study?". In a previous individual patient data (IPD) meta-analysis of over 30,000 patients from 6 trials, we found statistically significant subgroup effects in 2 trials .. but in opposite directions [1]. Identifying such IPD meta-analyses may be difficult when the individual trial subgroup effects were reported in 2007; one might need to perform a specific search for these studies, and consider multiple years?

Another point of interest is how often interaction with prognostic risk, e.g. from a multivariable regression model, was studied, rather than an interaction with a single characteristic. Such an approach was emphasized by Pocock and Lubsen in 2008 [2], but earlier examples may well be available. For example, a similar relative effect of accelerated tissue plasminogen activator (TPA) versus streptokinase treatment was found across the full range of mortality risk from acute myocardial infarction in the GUSTO-I trial, where the authors based risk on a multivariable combination of 5 baseline characteristics in a logistic regression model [3].

I like to encourage the authors to consider these 2 specific points in their review, or perform specific studies if necessary.

References
1. Hernandez AV, Westerhout CM, Steyerberg EW, Ioannidis JP, Bueno H, White H, Theroux P, Moliterno DJ, Armstrong PW, Califf RM et al: Effects of platelet glycoprotein IIb/IIIa receptor blockers in non-ST segment elevation acute coronary syndromes: benefit and harm in different age subgroups. Heart 2007, 93(4):450-455.
2. Pocock SJ, Lubsen J: More on subgroup analyses in clinical trials. N Engl J Med 2008, 358(19):2076; author reply 2076-2077.
3. Califf RM, Woodlief LH, Harrell FE, Jr., Lee KL, White HD, Guerci A, Barbash GI, Simes RJ, Weaver WD, Simoons ML et al: Selection of thrombolytic therapy for individual patients: development of a clinical model. GUSTO-I Investigators. Am Heart J 1997, 133(6):630-639.

Protocol amendment to be implemented shortly

Peter Sandercock — 2009-10-16T17:06:50Z

This comment is for information only

Protocol amendments. A number of minor changes to the IST-3 protocol document have been submitted for ethical and regulatory approval. The amended main trial protocol (and a supplementary protocol to describe the planned analyses of the cerebral perfusion data) will be posted on the IST3 website (www.ist3.com) when all the appropriate approvals have been obtained, in the near future.

Trial progress: By 16th October 2009, 1862 patients had been recruited, and the trial is on track to meet the target of 3100 patients by mid 2011.

For up to date information on recruitment and the implementation of the amendments please see www.ist3.com.

The Pan African Clinical Trials Register (PACTR_: Compliance with the WHO minimum data-set in the first Pan African WHO-endorsed Primary Registry

Amber Abrams — 2009-10-01T16:16:55Z

In the article “Compliance of clinical trial registries with the World Health Organization minimum data set : a survey” Moja et. al. explains that in September 2004, the International Committee of Medical Journal Editors (ICMJE) proposed “comprehensive trials registration as a solution to the problem of selective awareness and announce[d] that all 11 ICMJE member journals will adopt a trials-registration policy to promote this goal” (http://www.icmje.org/clin_trial.pdf). Thus, clinical trials must be publicly registered before the first participants are enrolled if the intention of the investigators is to publish findings in a medical journal.

To this end and to increase transparency and provide local registration opportunities for researchers who conduct clinical trials in Africa, the South African Cochrane Centre (SACC), based at the South African Medical Research Council, established a clinical trials registry for trials on HIV/AIDS, tuberculosis and malaria in 2007 (ATM Clinical Trials Registry). Since June 2009, this registry has expanded to contain clinical trials on all diseases, transitioning from the ATMCTR to the Pan African Clinical Trials Registry (PACTR). The PACTR is funded by the European and Developing Countries Clinical Trials Programme (EDCTP) and is managed in partnership with the Cochrane Infectious Disease Review Group and the Cochrane HIV/AIDS Review Group.

According to Moja et.al., although the World Health Organization (WHO) has created a mandatory minimum data-set as a means of standardizing available information “individual registry entries are largely incomplete”. Thus, the WHO has developed criteria by which a working group reviews applications of trial registries, and only after a stringent review to ensure all minimum standards are met, do they endorse a registry. Gaining a WHO primary status endorsement is a lengthy process as their working group is often developing new minimum standards leaving applicants for primary registry status responsible to meet these changing standards – this ensures that those registries with WHO endorsement are rigorous in their review of applications to their registries, and do not accept trials with incomplete data-sets.

Thus, the PACTR’s application to the WHO for primary status has been an ongoing process which was successful in June 2009 with the announcement that the PACTR“has been accepted by the Assistant Director General of the Information, Evidence and Research cluster” as a primary trials registry. PACTR will feed data into the global WHO International Clinical Trials Registry Platform (ICTRP) Search Portal, facilitating African representation in the global picture of planned, ongoing and completed clinical trials. Researchers conducting clinical trials in Africa who use PACTR as their registry of choice will now meet international requirements for research transparency.

Since registries “provide an essential tool to assess completeness of the information about all initiated trials” and are the “necessary first step to enable identification of all trials and the subsequent tracking of their results” (Moja et.al., http://www.trialsjournal.com/content/10/1/56 ), having an African based registry will benefit both Africa and the world as PACTR will actively encourage trial registration. Registering a clinical trial greatly increases exposure of the trial and assists researchers in determining gaps in research, reducing the likelihood of duplicated research and wasted funding and man-power (http://www.icmje.org/clin_trial.pdf).

The intention is for PACTR to become the single reference point for clinical trials activities in Africa feeding directly into the global WHO International Clinical Trials Search Portal, ensuring African representation in this global search engine. Additionally, PACTR’s database will include regularly downloaded information from the South African National Clinical Trials Registry (SANCTR) and will continue to build information sharing networks with national or other localized registries (Grobler et al, 2008:1201 – 1202). To learn more about PACTR or to register a trial, please visit our website at www.pactr.org.

Citation:
Grobler L, Siegfried N, Askie L, Hooft L, Tharyan P, Antes G. 2008. “Are national and multinational prospective trial registers necessary?” The Lancet, Oct 4;372(9645): pp. 1201-2.