Trials - Latest Comments

Response to comment by Hamilton et al.

Andreas Lundh — 2012-11-22T11:02:20Z

`Owing to technical issues with the Trials comments page, the authors were unable to post this comment when originally submitted on the 5th October¿

We acknowledge the various guidelines and ethical codes the medical writers' organisations have published, but wish to emphasize that no amount of guidelines and ethical codes can prevent misconduct. The drug industry also has ethical codes, e.g. the Pharmaceutical Research and Manufacturers of America (PhRMA) claims its members are ¿committed to following the highest ethical standards as well as all legal requirements.¿ (1) And its Code on Interactions with Healthcare Professionals states that: ¿Ethical relationships with healthcare professionals are critical to our mission of helping patients ¿ An important part of achieving this mission is ensuring that healthcare professionals have the latest, most accurate information available regarding prescription medicines.¿ (1) The disconnect between these lofty proclamations and the reality of big pharma¿s conduct is vast, which one can see by Googling the names of the largest companies together with "fraud" or "crime." It is also clear, e.g. from many internal industry documents released because of court cases (2,3), that what we say is true: If medical writers don't do a job that satisfies the sponsor's marketing department, they might go out of business. In industry trials, medical writing is done to satisfy the sponsor's marketing needs, no matter whether it is a clinical trial, a review, or an opinion piece that is being written up.

Andreas Lundh, Lasse T. Krogsboll, Peter C. Gotzsche
The Nordic Cochrane Centre, Rigshospitalet

References
1. PhRMA Code on Interactions with Healthcare Professionals. [http://www.phrma.org/sites/default/files/108/phrma_marketing_code_2008.pdf] (accessed 5 October 2012).

2. Drug Industry Document Archive. [http://dida.library.ucsf.edu] (accessed 5 October 2012).

3. Wyeth Ghostwriting Archive. [http://www.plosmedicine.org/static/ghostwriting.action] (accessed 5 October 2012).

Addressing the implication made by the authors that if medical writers "do not do a job that satisfies the sponsors¿ marketing department, they might go out of business."

Cindy Hamilton — 2012-09-20T16:29:59Z

To the Editor:

Having read with interest the article by Lundh, Krogsboll, and Gotzsche: Sponsors¿ Participation in Conduct and Reporting of Industry Trials: A Descriptive Study (1), we would like to take this opportunity to address the implication made by the authors that if medical writers "do not do a job that satisfies the sponsors¿ marketing department, they might go out of business." This comment may be relevant for ghostwriters, but we do not believe it holds for professional medical writers. As leaders of the Global Alliance of Publication Professionals (GAPP), we believe it is important for your readers, and indeed Lundh et al, to recognize the difference between these two types of writers.

In our experience, professional medical writers have to satisfy authors, journal editors, peer-reviewers, and, within industry, the medical and compliance departments, not marketing departments. The European Medical Writers Association (EMWA), the American Medical Writers Association (AMWA), and the International Society for Medical Publication Professionals (ISMPP) have ethical codes of behavior for writers (2-4). Although the specific wording might vary among these codes, the core principles of preparing documents that are objective, accurate, scientifically valid, and complete are held in common. Further, international guidelines (5), published in 2009 and followed by pharmaceutical companies, specifically highlight that sponsors can provide scientific comments back to authors; the guidelines provide no role for marketing department staff. Notably, pharmaceutical companies are issuing publication policies that categorically prohibit marketing staff from being involved in manuscript preparation review and approval (6-10). Further, budgets for medical writing services are increasingly being held by medical departments, not marketing departments. If Lundh, et al want to gain direct insight into current ethical writing practices (rather than speculate on the business futures of ghostwriters), we encourage them to review these codes and policies, and certainly welcome them to contact GAPP.

Like Lundh et al, we abhor ghostwriting and always encourage authors to be transparent about medical writing assistance. If authors use writers, we encourage use of the "anti-ghostwriting checklist¿11. Incidentally, this checklist was included in a viewpoint series that involved leaders from GAPP and Peter Gotzsche, one of Lundh¿s co-authors ¿ we were surprised that this important tool was not referenced in the Lundh article.

We also suggest that readers refer to a more detailed response to the Lundh article, to be posted on the GAPP website: www.gappteam.org/

With kind regards, Art Gertel on behalf of fellow GAPP members Dr Cindy Hamilton, Dr Adam Jacobs, Gene Snyder, and Dr. Karen Woolley.

Disclosures: All GAPP members have held, or do hold, leadership roles at associations representing professional medical writers (eg, AMWA, EMWA, DIA, ISMPP, ARCS), but do not speak on behalf of those organizations. GAPP members have or do provide professional medical writing services to not-for-profit and for-profit clients.

REFERENCES:

1. Lundh A, Krogsboll L, Gotzsche P. sponsors¿ participation in conduct and reporting of industry trials: a descriptive study. Trials 2012, 13:146

2. American Medical Writers Association (AMWA): http://www.amwa.org/default.asp?id=114

3. European Medical Writers Association (EMWA): http://www.emwa.org/Home/Ghostwriting-Positioning-Statement.html

4. International Society for Medical Publication Professionals (ISMPP): http://www.ismpp.org/ISMPP%20CODE%20OF%20ETHICS%2011_10.PDF

5. Good Publication Policies 2 (GPP2). Graf C, Battisti WP, Bridges D, Bruce-Winkler V, Conaty JM, Ellison JM, Field EA, Gurr JA, Marx ME, Patel M, Sanes-Miller C, Yarker YE; International Society for Medical Publication Professionals. Research methods and reporting. Good publication practice for communicating company sponsored medical research: the GPP2 guidelines. BMJ 2009;339:b4330.

6. Merck Publication Policy: http://www.merck.com/research/discovery-and-development/clinical-development/Merck-Perspective-Clinical-Trials.pdf.

7. GSK Publication Policy: http://www.gsk.com/policies/GSK-on-disclosure-of-clinical-trial-information.pdf

8. Pfizer Publication Policy: http://www.pfizer.com/research/research_clinical_trials/registration_disclosure_authorship.jsp

9. PhRMA Clinical Trial Principles: http://www.phrma.org/sites/default/files/105/042009_clinical_trial_principles_final.pdf

10. Statements by Pharmaceutical Publication Planning Professionals at recent meeting of The International Publication Planning Association (TIPPA). Chicago, July 2012.

11. Gøtzsche PC, Kassirer JP, Woolley KL, Wager E, Jacobs A, et al. (2009) What Should Be Done To Tackle Ghostwriting in the Medical Literature? PLoS Med 6(2): e1000023. doi:10.1371/journal.pmed.1000023

Efficacy due to alcohol or something else in the mouthwash composition?

Dirk Lachenmeier — 2012-03-27T10:36:03Z

As it was previously noted, a considerable industry bias in mouthwash-related research exists [1]. Therefore, it would be interesting to know the sponsor of the study and if the sponsor was affiliated with any of the mouthwash brands under investigation.
To provide insight into the transferability of the results, data about the composition of the two mouthwash types under investigation would be important as the composition is highly variable between brands and types [2,3]. Did the two products have the same content of essential oils? What kinds of essential oils, which may have different efficacy dependent on type, were used? At least the manufacturer of the alcohol-containing products provides different product types with different composition, which one was it? Would it not have been a better experimental design to use the same formulation with and without alcohol than to use two commercial types of mouthwash, which may have additional differences other than the alcohol content? How can the conclusion that the difference is due to alcohol uphold, when the composition has not been characterized at all?

[1] Lachenmeier DW: Safety evaluation of topical applications of ethanol on the skin and inside the oral cavity. J Occup Med Toxicol 2008, 3:26. http://www.occup-med.com/content/3/1/26
[2] Lachenmeier DW, Keck-Wilhelm A, Sauermann A, Mildau G: Safety assessment of alcohol-containing mouthwashes and oral rinses. SOFW J 2008, 134:70-78.
[3] Lachenmeier DW, Gumbel-Mako S, Sohnius EM, Keck-Wilhelm A, Kratz E, Mildau G: Salivary acetaldehyde increase due to alcohol-containing mouthwash use: a risk factor for oral cancer. Int J Cancer 2009, 125:730-735.

Comparing apples and oranges

Joselita Salita — 2011-07-21T12:45:16Z

I appreciate the goals of Nairy et al.'s paper. However, their primary objective was incomplete which made them design an experiment which compared apples and oranges. The three groups could only be partially compared: group 1 vs 2 and 1 vs 3, which in the end, could not support their conclusions.

In order to assess the efficacy of in-situ gels in treating OPC, one compares in-situ gels with a placebo or a comparator (in this case, flucanozole tablets) or both among one patient type. Having two types of patients, with one type incompletely represented was a major flaw. The choice of comparing responses among HIV/AIDS patients as a patient type was also not a good idea due to confounding effects. The resulting „good clinical response“ of HIV patients after 21 days in which 85% in Group I receiving in situ gels were considered OPC cured compared to 66% HIV patients in Group III treated with tablets could be insignificant not only due to small sample size (n=15) but also to confounding effects. So did relapse of 80 and 90% respectively. Group II was comprised of non-HIV patients and was treated with in situ gels. Better outcomes for this group (clinical response=100% and relapse=0) may have resulted because the subjects were HIV-free (therefore responded better) and similar results could have been obtained if this group received placebo in-situ gel or flucanozole tablets. For this particular data set no statistical tests were performed but comparison was not valid anyway.

All other data with or without statistical support showed the same trends, that is, Group II having the most positive outcome and Group III the least. I think they have no relevance under the same reasoning. The only valid conclusion in this paper is that in situ gels are more efficacious in patients without HIV/AIDS than those with HIV/AIDS. Whether in situ gels are better than tablets among healthy or HIV patients is unanswered.

To the Editor

Dirk Kuhlmann — 2011-06-23T12:09:45Z

The study showing an in situ gel formulation is an interesting approach for treatment of OPC. The clinical study data presented by Nairy et al. is poorly organized and confusing.

The reliability of the results is questionable due to incomplete outcome data and selection bias from randomization based on clinical evaluation. The three study groups are only partially comparable. As group II is missing a control group, the data is not supportive of the aim of the study.

Severity of signs and symptoms are mild to absent according to table 3. An effect of treatment on moderate to severe cases is not presented. Patient demographics of group 3 are diverse with a possible confounding bias associated with a large variance in CD4 count.

The primary endpoints are not specified to be at EOT or after follow-up. Data and results presented throughout the paper are not consistent thereby decreases their validity. Gender counts for group II vary in the study design and table 2. Changes in symptoms vary throughout the paper. In the abstract, 97% of group I were cured. In the discussion, 95% of this group were cured but only 80% showed a good clinical response in the in vivo evaluation up to 21 days. At EOT, group III showed 85% improvement in the abstract and 93% and 66.6% as cured in the discussion section and in vivo evaluation respectively. The results in figure 2 are not supportive of other data. Percentage points do not correlate to natural numbers of patients.

In group I and II, 40% of the patients experienced gastrointestinal symptoms. Unfortunately no mention of adverse events is made for the control group.

According to the authors, there is no significant difference in clinical response at EOT and at end of follow-up and no improvement in group I and III at the end of the study.
In my opinion, the data is not supportive for concluding a better clinical efficacy in HIV/AIDS patients. Successful clinical outcome was only reported during the treatment duration. Insufficient information is given to discern whether the in situ gel is a viable alternative to conventional formulations.

It would be advantageous if the authors could repeat the study in a larger, randomized patient group with a suitable control group and study setup and with treatment endpoints comparable to other OPC studies.

Yours sincerely
D. Kuhlmann

Systematizing the analysis of effect heterogeneity requires rethinking some fundamentals

James Scanlan — 2011-06-01T10:30:15Z

The article by Gabler et al.[1] questions the soundness of epidemiological literature’s reporting and analysis of heterogeneity of treatment effects (HTE) and calls for greater attention to HTE issues and more systematic analysis of such issues.

But the goals the authors seek cannot be achieved without reconsideration of certain fundamentals of subgroup analysis. Standard approaches to such analyses are based on an assumption that absent HTE all subgroups will experience equal proportionate changes in outcome rates (i.e., the same rate ratio across different baseline rates) and that HTE is observed in those cases where equal proportionate changes are not found. That assumption, however, is demonstrably unsound for the simple reason that it is not possible for a factor to cause equal proportionate changes in an outcome while causing equal proportionate changes in the opposite outcome.

That is, for example, if Group A has a baseline rate of 5% and Group B has a baseline rate of 10%, a factor that reduces the two rates by equal proportionate amounts, say 20% (from 5% to 4% and from 10% to 8%) would necessarily increase the opposite outcome by different proportionate amounts (95% increased to 96%, a 1.05% increase; 90% to 92%, a 2.2% increase). And since there is no more reason to expect that two group would undergo equal proportionate changes in one outcome than there is to expect they would undergo equal proportionate changes in the opposite outcome, there is no reason to expect that the two groups would undergo equal proportionate changes in either outcome.

For reasons inherent in the shapes of normal distributions of factors associated with experiencing or avoiding an outcome, it is more reasonable to expect that a treatment that reduces an outcome rate will tend to cause a larger proportionate decrease in that outcome for groups with the lower base rates while causing a larger proportionate increase in the opposite outcome for other groups.[2-6]

These considerations raise several issues for subgroup analyses. The first involves identifying benchmarks across baseline rates the departure from which would be deemed a subgroup effect. See Table 6 of the presentation in reference 4 for the application of the approach to identifying subgroup effects discussed in reference 4 to data in reference 49 of Gabler et al. A second, and more important, involves using information derived from observed treatment effects for particular groups to estimate absolute risk changes in groups for which treatment effect information is not available or not reliable.[5,6]

References:

1. Gabler NB, Naihua D, Liao D, et al. Dealing with heterogeneity treatments: is the literature up to the challenge. Trials 2009,10:43: http://www.trialsjournal.com/content/10/1/43 (Accessed May 29, 2011.)

2. Scanlan JP. Race and mortality. Society 2000;37(2):19-35: http://www.jpscanlan.com/images/Race_and_Mortality.pdf (Accessed May 29, 2011.)

3. Scanlan JP. Divining difference. Chance 1994;7(4):38-9,48: http://jpscanlan.com/images/Divining_Difference.pdf (Accessed May 29, 2011.)

4. Scanlan JP. Interpreting Differential Effects in Light of Fundamental Statistical Tendencies, presented at 2009 Joint Statistical Meetings of the American Statistical Association, International Biometric Society, Institute for Mathematical Statistics, and Canadian Statistical Society, Washington, DC, Aug. 1-6, 2009: PowerPointPresentation: http://www.jpscanlan.com/images/Scanlan_JSM_2009.ppt; Oral Presentation: http://www.jpscanlan.com/images/JSM_2009_ORAL.pdf (Accessed May 29, 2011.)

5. Scanlan’s Rule page of jpscanlan.com: http://jpscanlan.com/scanlansrule.html (Accessed May 29, 2011.)

6. Scanlan JP. Assessing heterogeneity of treatment effects in light fundamental statistical tendencies. Trials May 26, 2011(responding to Kent DM, Rothwell PM, Ionnadis JPA, et al. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials 2010,11:85): http://www.trialsjournal.com/content/11/1/85/comments#498686 (Accessed May 29, 2011.)

Assessing heterogeneity of treatment effects in light fundamental statistical tendencies

James Scanlan — 2011-05-26T15:41:35Z

The article by Kent et al.[1] provides useful guidance on the reporting of results by subgroup. But the article suffers from the common assumption that the absence of a subgroup effect (heterogeneity) is reflected by equivalent relative risk reductions across subgroups. It is not logical to regard equivalent relative risk reductions as somehow normal (i.e., as reflecting the absence of a subgroup effect) for the simple reason that it is not possible for a factor to cause equal relative reductions in an outcome for groups with different base rates while causing equal relative increases in the opposite outcome for those groups.

The point can be illustrated with figures in Table 1 of Kent et al., which shows the same 25% relative risk reduction of an adverse outcome for an average risk group (4% reduced to 3%) and a high risk group (20% reduced to 15%). Such figures would mean that the increase in the favorable outcome rate was 1% for the low risk group (96% increased to 97%) but 6.3% for the high risk group (80% increased to 85%). Since there is no more reason to regard it as somehow normal that there will be equal relative changes in one outcome than to regard it as somehow normal that there will be equal relative changes in the opposite outcome, there is no reason to regard it as somehow normal that there will be equal relative changes in either outcome.

In fact, for reasons related to the shapes of normal risk distributions there is reason to expect that a factor that reduces the risk of an outcome will tend to cause groups with lower base rates to experience larger relative reductions in those rates, while causing other groups to experience larger relative increases in the opposite outcome rates.[2-7] Thus, reliance on relative risk as a measure of effect size tends to yield opposite conclusions as to whether high or low risk groups derive the greater benefit from an intervention depending on whether one examines the favorable or the adverse outcome.

The only way to determine whether there exists a meaningful subgroup effect is to derive from the rates for controls and treated subjects in each subgroup the differences between means of the hypothesized underlying distributions.[2-4,7] Taking the data from Table 1 of Kent et al., the reduction from 4% to 3% for the average risk group reflects a differences between means of the underlying distributions of .13 standard deviations. An equivalent reduction in the risk for the high risk group would be reflected by a reduction from 20% to 17%. Determinations of meaningful interaction should be based on departures from patterns such as these, not departures from equivalent relative changes in either the adverse or the favorable outcome. Similarly, appraisals of the likely absolute risk reduction for each subgroup – as Kent et al. note, the key clinical consideration – should be based on benchmarks derived as just described rather than on benchmarks based on equivalent relative risk reductions.

References:

1. Kent DM, Rothwell PM, Ionnadis JPA, et al. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials 2010,11:85: http://www.trialsjournal.com/content/11/1/85 (Accessed May 1, 2011.)

2. Scanlan JP. Interpreting Differential Effects in Light of Fundamental Statistical Tendencies, presented at 2009 Joint Statistical Meetings of the American Statistical Association, International Biometric Society, Institute for Mathematical Statistics, and Canadian Statistical Society, Washington, DC, Aug. 1-6, 2009: http://www.jpscanlan.com/images/JSM_2009_ORAL.pdf;http://www.jpscanlan.com/images/Scanlan_JSM_2009.ppt (Accessed May 1, 2011.)

3. Scanlan JP. Rethinking the premises of subgroup analyses. BMJ June 7, 2010 (responding to Sun X, Briel M. Walter SD, and Guyatt GH. Is as subgroup effect believable? Updating criteria to evaluated the credibility of subgroup analyses. BMJ 2010;340:850-854): http://www.bmj.com/cgi/eletters/340/mar30_3/c117 (Accessed May 1, 2011.)

4. Scanlan JP. Problems in identifying interaction where groups have different base rates. BMJ Sept. 21, 2010 (responding to Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ 2003;326:219): Altman DG, Bland JM. Interaction revisited: the difference between two estimates. BMJ 2003;326:219): http://www.bmj.com/content/326/7382/219/reply#bmj_el_241943 (Accessed May 1, 2011.)

5. Scanlan JP. Race and mortality. Society 2000;37(2):19-35: http://www.jpscanlan.com/images/Race_and_Mortality.pdf (Accessed May 1, 2011.)

6. Scanlan JP. Divining difference. Chance 1994;7(4):38-9,48: http://jpscanlan.com/images/Divining_Difference.pdf (Accessed May 1, 2011.)

7. Subgroup Effects sub-page of Scanlan’s Rule page of jpscanlan.com: http://www.jpscanlan.com/scanlansrule/subgroupeffects.html (Accessed May 1, 2011.)

Erratum: minor error

Andrew Vickers — 2010-11-30T09:14:31Z

In the text of our protocol, we gave an incorrect affiliation to JianPing Liu. He is not, as we suggested, affiliated with the Chinese Cochrane Center, but is Director of the Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine. The last sentence of the "Search strategy for identification of studies" section should therefore read:

Chinese trials will be identified by a separate process: Jianping Liu of the Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, will use that institution's resources to identify Chinese trials of acupuncture for chronic pain that involved full allocation concealment.

Educational opportunities available to Trial and Data Managers

Rachel Breen — 2010-09-01T10:04:55Z

I read with interest the article (provisional pdf) regarding Trial Management.
Managing clinical trials
Trials 2010, 11:78 doi:10.1186/1745-6215-11-78
Barbara Farrell, Sara Kenyon, Haleema Shakur.

However I felt that the article did not provide a balanced review of the educational opportunities available to Trials Managers and Data Managers. The article only mentioned the London School of Hygiene and Tropical Medicine's MSc in Clinical Trials. There are other MSc courses that have been developed to address trial management needs and could be argued to be more specific to the needs of Trial Managers than LSHTM. Examples include the Cranfield MSc and also MSc Clinical Research Administration run by the University of Liverpool, which is conducted completely in an online learning environment and was developed to meet the needs of Trials Managers in particular. This MSc encourages students to undertake research dissertations relevant to the practices of trial management. Typical examples of dissertation projects include: core outcome sets (an agreed standardised minimum set of outcomes that should be measured and reported in all clinical trials of effectiveness in a specific condition), consent issues, data capture, ethics committees in multicentre/multinational studies, non-response bias, study recruitment, trial monitoring, adherence to trial reporting standards and issues of relevance to the pharmaceutical industry or specific clinical areas.
This MSc is also linked to the activities of the MRC NWHTMR (www.methodologyhubs.mrc.ac.uk) in part due to the relevance of the dissertation projects. I would also recommend that interested readers contact the MRC Hub network to pursue trial management research.

Searching for new insights in subgroup analyses

Ewout Steyerberg — 2009-11-24T12:50:11Z

The SATIRE group is to be applauded for their effort to shed more light at the minefield of subgroup analysis, reporting, and interpretation. The question may however be what truly new insights will be obtained. One might predict that findings from a well-selected set of papers published in 2007 will be largely similar to what was found before in the studies shown in Table 1.

Of particular interest may be the answer to question 10 in Appendix 1: "Is the interaction consistent across closed related outcomes within the study?". In a previous individual patient data (IPD) meta-analysis of over 30,000 patients from 6 trials, we found statistically significant subgroup effects in 2 trials .. but in opposite directions [1]. Identifying such IPD meta-analyses may be difficult when the individual trial subgroup effects were reported in 2007; one might need to perform a specific search for these studies, and consider multiple years?

Another point of interest is how often interaction with prognostic risk, e.g. from a multivariable regression model, was studied, rather than an interaction with a single characteristic. Such an approach was emphasized by Pocock and Lubsen in 2008 [2], but earlier examples may well be available. For example, a similar relative effect of accelerated tissue plasminogen activator (TPA) versus streptokinase treatment was found across the full range of mortality risk from acute myocardial infarction in the GUSTO-I trial, where the authors based risk on a multivariable combination of 5 baseline characteristics in a logistic regression model [3].

I like to encourage the authors to consider these 2 specific points in their review, or perform specific studies if necessary.

References
1. Hernandez AV, Westerhout CM, Steyerberg EW, Ioannidis JP, Bueno H, White H, Theroux P, Moliterno DJ, Armstrong PW, Califf RM et al: Effects of platelet glycoprotein IIb/IIIa receptor blockers in non-ST segment elevation acute coronary syndromes: benefit and harm in different age subgroups. Heart 2007, 93(4):450-455.
2. Pocock SJ, Lubsen J: More on subgroup analyses in clinical trials. N Engl J Med 2008, 358(19):2076; author reply 2076-2077.
3. Califf RM, Woodlief LH, Harrell FE, Jr., Lee KL, White HD, Guerci A, Barbash GI, Simes RJ, Weaver WD, Simoons ML et al: Selection of thrombolytic therapy for individual patients: development of a clinical model. GUSTO-I Investigators. Am Heart J 1997, 133(6):630-639.