Why coronavirus vaccine trials don’t have all the answers

There are currently at least six Covid-19 vaccines in phase 3 clinical trials – the final phase of testing. These trials all aim to compare the safety and effectiveness of the vaccines versus a placebo. However, as Peter Doshi, associate editor at the BMJ, asks in a new report, what does “effective” actually mean?

The primary goal of these phase 3 trials in progress is to determine whether the vaccine reduces the risk of a person getting symptomatic Covid-19. To be counted as a Covid-19 case, trial participants must have a positive swab test, as well as a defined list of symptoms – which varies from one trial to another. These symptoms can range from a mild headache through to severe disease requiring intensive care.

Each trial uses their own definition of a positive case to estimate how many people are expected to get Covid-19 in the control group (those not receiving the experimental vaccine). For example, the Moderna vaccine clinical trial protocol works on the assumption that one out of 133 people will develop symptomatic Covid-19 over a six-month period. If the vaccine is 60 per cent effective, then complex statistical analysis dictates that only 151 people out of 30,000 recruits need to become symptomatically infected for this degree of protection to be apparent.

Concern has been raised that by following this type of trial design, it won’t be possible to tell whether a vaccine protects against severe disease or death. Indeed, the design of these first trials does not differentiate mild from severe cases of Covid-19 in the primary analysis, but there are very good reasons for this and it should not be a cause for alarm.

Quite simply, many fewer people die from Covid-19 than develop mild symptoms of disease. To prove that a vaccine protects against only severe or fatal cases would require many more people to be recruited to each trial. With trials already involving tens of thousands of participants, this is just not realistic at this stage.

Trials testing severe disease or death alone as an endpoint would need much more time and money to be completed. So designing these first phase 3 trials has been a balancing act: being able to show whether some degree of protection is achieved while delivering these results in the most timely manner.

Also, whereas the severity of disease is not the focus of trial outcome, all ongoing trials are still carefully monitoring the severity of all Covid-19 cases. Valuable conclusions can still be drawn from this data, even if statistical significance cannot be proved.

Another issue that has been raised regarding current phase 3 clinical trials has been the fact that the people who most need protecting, such as the elderly and those with compromised immune systems (such as people undergoing chemotherapy), are not being recruited. But this is a standard recruitment approach for any clinical trial, so it is not unexpected. It does mean that the conclusions drawn about vaccine efficacy may not directly apply to those people excluded from trials. Still, a vaccine that can reduce symptomatic Covid-19 in healthy adults is essential, as it will reduce the risk of infection of vulnerable group.

It is important to be aware of the limitations of the current trials, but these should not be considered as major flaws. The aim of any clinical trial is to examine a subset of the population to make the best guess as to what will happen if the entire population is treated the same way.

Ultimately, only when the whole population is vaccinated can the exact efficacy of a vaccine be determined. Coronavirus vaccine trials are, therefore, expected to continue for years to come, each one contributing to our understanding of how to control this virus.

Sarah L Caddy is a clinical research fellow in viral immunology and a veterinary surgeon at the University of Cambridge. This article first appeared on The Conversation