By Ray Tobey, Chief Medical Officer on Apr 14, 2017 12:14:05 AM
The short answer is limited. Good information can be gained from a pilot study but not at a volume that draws significant conclusions. Let's discuss pros and cons.
Con: Not Enough Statistical Power
The idea of a pilot study is to test it out in a small number of patients with the target disease, say 10-15, to see how they respond. Maybe a few patients will be given a placebo. If a statistician is involved in your study design, I would expect her to say it is very unlikely that any results, positive or negative, will be suggestive let alone definitive. It simply lacks statistical power for valid group comparisons. As a result, it does not give you any solid evidence for "go or no go" decisions. If that's true, why bother with it?
Pro: Learn Small Improvements Needed
In a pilot study, you should get specific data that is very useful in designing future trials by learning the degree of positive improvements, the variability of those changes and some degree of comparison to your placebo patients responses. For a new agent or class of drugs, this can be vital for guidance in future designs.
Pro: Learn More About Adverse Effect
It also may give you some information on adverse effects not suspected from animal studies. We have no way of learning about the onset of headaches from rodent studies, for example. Other adverse effects may become apparent. e.g., gastrointestinal-intestinal discomfort or skin rashes. This information can be vital for designing future trials and decision making.
Pro: Make Adjustments to Dose Selections
If from laboratory data you have an estimate of drug concentration for cellular or organ effectiveness, you may be able to estimate drug doses to achieve those blood or tissue levels from administrations to a pilot group of patient. Presumably, you already have some pharmacokinetic data in your previous safety studies in humans but a pilot study in patients may give you different dosing information.
Pro: Pilot Studies Can Be Done Without An IND Application
It's possible within certain regulatory guidelines for a pilot study to be done without an IND (Investigative New Drug) application. Generally, this requires that the drug is manufactured or prepared within the state where the study will be performed. It still requires state department of health approval that is generally less costly and quicker than an FDA IND approval.
Con: Doesn't Inform Decision Making Enough
Although there can be some worthwhile advantages to conducting pilot studies, it doesn't usually gain you much for the drug development timeline. It also doesn't always gain the solid evidence to inform management's "Go or No-Go" decision, which was your primary reason for doing a pilot study in the first place. However, the data obtained may still be useful for securing funding for future studies.
Good science requires us to remain skeptical of a process for the sake of process, and focus on the reasoning behind a decision or project. Asking "are pilot studies actually useful?" is an important exercise for evaluating their place in the clinical research process.
Clinica development teams often use pilot studies in hopes of obtaining decision-informing data around a new compound. Yet, the usefulness of the data produced in a pilot study may fall short of this goal. Still, there are many secondary benefits produced in a pilot study that can save a team from making mistakes later in the clinical research process when the consequences are much more expensive; dosage or adverse events expectations are two good examples. Overall, with pilot studies, the pros outweigh the cons.
ClinSmart has a great deal of experience in designing and implementing pilot studies and for the reasons above we focus pilot studies on mitigating the shortcoming of obtained data while maximizing the secondary benefits that can be so useful.