Clinical Trials Registry Shows Its Strengths, Flaws

J U LY 2 0 1 2 • G I & H E PATOLOGY NEWS COMMENTARY

Download PDF
.
Clinical Trials Registry Shows Its Strengths, Flaws

BY DR. ROBERT SOLOMON

Fifteen years ago, the U.S. Food and Drug Administration established a registry for clinical trials, www. clinicaltrials.gov, so that scientific investigators and the general public would have a ready source of information about medical research – planned, ongoing, and completed.

In 2004, the International Committee of Medical Journal Editors (ICMJE) announced that their journals would publish the results of trials only if they had been registered.

Such a registry is of great potential value to medical scientists who wish to learn whether others are pursuing similar lines of investigation, facilitating their efforts to learn from others’ work even when it has not been published, and potentially enabling collaboration. In the May 2, 2012, issue of JAMA, Dr. Robert M. Califf and colleagues report their analysis of trials entered in the registry, focusing on cardiovascular, oncology, and mental health research.

Their findings reveal some causes for concern, but also can be looked upon as opportunities for improvement. While there was an increase in early registration of trials between the two periods of this analysis (2004-2007 and 2007- 2010), fewer than half of trials are registered before enrollment of participants. This directly thwarts one of the major goals of the registry: transparency.

Failure to register trials before they are underway suggests that some studies may not be registered if their findings are negative or otherwise unsatisfactory to investigators, sponsors, or sources of funding. A comprehensive registry can overcome publication bias – the tendency for studies with negative or unwelcome findings never to appear in print – only if trials are uniformly entered.

The study authors found a remarkable proportion of trials to be small and conducted at single sites ( JAMA 2012;307: 1838-47). This raises concerns about statistical power and external validity. A move toward larger, multicenter trials, which should be enhanced by a comprehensive registry, could effectively address both of these concerns. In the hierarchy of evidence-based medicine, the randomized, double-blind, prospective trial is at the top of the heap, so to speak, with respect to desired characteristics for an original investigation.

Yet only about a third of trials were randomized, with a similar proportion double- blinded. While it is very difficult to conduct some studies with randomization and blinding, the failure to do so introduces major sources of potential bias and confounding, and statistical manipulations can never completely adjust for these. Such disappointing numbers raise serious doubts about the quality of scientific investigation.

Sponsorship is a significant concern, as its influence on study design, interpretation of results, and likelihood of publication if hoped-for outcomes are not obtained has been long established The numbers in this analysis indicate that nearly 4 in 10 registered trials had industry sponsorship.

That a major source of potential bias should be present in such a large proportion of investigations is troubling. At a time when the Obama administration is emphasizing the need for a robust approach to comparative effectiveness research so that doctors and patients can make the best decisions and ensure a high quality of health care, it is disconcerting – to say the least – that only 4% of clinical trials had the National Institutes of Health or other federal agencies listed as lead sponsors. Public funding of this research must expand dramatically.

Finally, from a health policy perspective, it is disappointing to note that only about 10% of trials were focused on prevention, and only about 2% in the area of health services. If we are to make our health care system more effective and efficient, this most certainly represents an opportunity to do better by doing more. ■

ROBERT C. SOLOMON, M.D., is core faculty in the emergency medicine residency at Allegheny General Hospital, Pittsburgh; Assistant Professor in the department of emergency medicine at Temple University, Philadelphia; and Medical Editor in Chief of ACEP NEWS.


P E R S P E C T I V E


PAUL MOAYYEDI, MB CHB, PH.D., MPH, FRCP, FRCPC, AGAF, FACG, is Director of the Division of Gastroenterology, McMaster University, Hamilton, Canada

Clinicaltrials.gov was released to the public in February 2000 and has been a tremendous success story. The United States has provided the world with a trials registration website that has over 124,000 trials registered since its inception.

Although there are other trial registration sites available, clinicaltrials.gov accounts for approximately 80% of randomized trials on the World Health Organization (WHO) portal – a collection of all trial registries known to WHO.

This has allowed patients to know what trials are going on for their disease and to volunteer for the study if that is appropriate.

The registry also reduces the possibility that negative trials will not be published, as the study is now in the public domain and so anyone can ask what has happened if nothing is heard once enrollment is completed.

Systematic reviewers can also collect information on ongoing studies that might change the conclusions of their review as well as chase up those studies where data isn’t being made publicly available. It is a sad reflection of human psychology that we don’t focus on the progress we have made, but rather emphasize the deficiencies of the system we’ve created.

There have been a number of papers outlining the problems with trials that are registered on clinicaltrials.gov, and the latest example of this is a report by Dr. Califf and colleagues published in JAMA (2012;307:1838-47).

The authors evaluated 96,346 cardiovascular, mental health, and oncology studies that were registered with clinicaltrials.gov from 2000 to 2012. In almost two-thirds of these studies, the principal investigator originated from the United States, and 63% of the studies were drug trials. There were many problems with the studies that were registered, such as 66% being single-center studies, only 40% having a data monitoring committee, and almost 60% being underpowered, intending to enroll fewer than 100 patients. Time trends do not suggest that the situation is improving.

The NIH only funded a small proportion of studies (10%), and this support is declining with time (only 3% of studies registered in 2007- 2010 were funded by the NIH). Industry was the main supporter of trials, with 32% of all trials (and 59% of all participants in trials) funded by pharmaceutical and device companies.

The statistics certainly tell us there is room for improvement. Not least is that we need greater government funding of randomized controlled trials rather than less, as seems to be happening at the moment.

There is also a great deal of information that we would like from the database but cannot obtain. It would be better if we had more information, and I hope those that run clinicaltrials.gov make more fields mandatory so we can have more information about ongoing trials in the future. However, I would like to dwell on the positive message from these data.

The fact is that we only know of the deficiencies in what we are doing because so many people are registering their trials.

There was a great deal of resistance to the concept behind clinicaltrials.gov from some academics and many pharmaceutical companies when this was first proposed.

In 12 years we have come a long way, and at least we have a record of things we are doing well and things that we could improve upon. We will progress rapidly in the next 12 years if only we can learn from the lessons these data teach us.

PAUL MOAYYEDI, MB CHB, PH.D., MPH, FRCP, FRCPC, AGAF, FACG, is Director of the Division of Gastroenterology, McMaster University, Hamilton, Canada

http://www.gastro.org/journals-publications/gi-hepatology-news/GIHEP_July_2012_LoRes_AGA_Nxtbooks.pdf