CRASH 3: TXA for traumatic brain injury fails the EBM 2.0 test

CRASH 3 was released in 2019 to much fanfare and controversy. The trial sought to answer the question of whether Tranexamic Acid (TXA) provides benefit in blunt traumatic brain injury (TBI).

Here we will review this paper using the EBM 2.0 approach – it is best to review EBM 2.0 first if you have not already, in order to properly appreciate this analysis.

Pre-trial probability of hypothesis

On the face of it, there is some biological plausibility for believing this could work. Patients with brain injuries die from different mechanisms, but one of those mechanisms is certainly the effect of bleeding within a confined space (the skull) so it is conceivable that a drug that stabilises blood clots may reduce death via this mechanism. However it is not plausible that TXA would reduce death caused by other mechanisms (e.g. diffuse axonal injury) and so any potential overall benefit would be likely through reducing deaths only within the sub-group of patients who died from bleeding within the skull, without a substantial increase in harm to the whole group.

Unfortunately, the TICH-2 trial in 2018 of TXA for primary intracranial haemorrhage (ICH) did not suggest even the smallest of benefits for TXA and in the old EBM 1.0 world, where we talked about positive and negative trials, this one was “very negative” (in the EBM 2.0 world there are no positive or negative trials, just different levels of post-trial hypothesis uncertainty). There was no benefit indicated in either the primary outcome of disability (measured by modified Rankin Scale) or the secondary outcome of mortality and there was little to get excited about even for further study amongst the subgroups. Of note regarding haematoma size, there was a very small 1.2mL difference in haematoma expansion (p=0.03)  but evidently this didn’t translate to any meaningful patient oriented outcomes. A good discussion of the trial’s strengths and limitations are provided by First10em and Rebelem.

Now, unlike TBI, people who die from ICH almost exclusively die from the bleeding itself so should be the most likely intracranial pathology to benefit from TXA. Consequently, based on these results, the pre-trial probability for TXA having a benefit in a condition such as TBI where patients die from other causes, not just bleeding, is arguably very close to zero.

There was one other small trial (238 patients) by Yutthakasemsunt 2013 that studied TXA in TBI investigating the primary outcome of “progressive intracranial haemorrhage” defined as intracranial haemorrhage seen on the second CT scan that was not seen on the first CT scan, or an intracranial haemorrhage seen on the first scan that had expanded by 25% or more on any dimension (height, length, or width) on the second scan. This outcome was seen in 21 (18%) patients allocated to TXA and in 32 (27%) in the placebo group with a RR = 0.65 (95% CI 0.40 to 1.05). In considering these results from a single small study with the wide CI crossing 1.0 (with implied relatively high p value), it is thus quite unclear whether this is a real finding at all and even if it was, very unclear if this would have a relevant impact on mortality. This is particularly the case for a condition where bleeding expansion is just one of several reasons for death, when in TICH-2 examining the condition that is essentially the only reason for death (ICH), reducing change in haematoma size didn’t improve outcomes.

So the pre-trial probability of benefit is probably not sufficiently raised from near zero by the consideration of Yutthakasemsunt 2013, however to be generous, for the purpose of this analysis, lets make it 5%. 

Bias

CRASH 3 was subject to bias which makes any study finding very unlikely to be true – an excellent discussion of this trial is provided here by First10em. Notably the primary outcome from the pre-registered trial of “all cause mortality” was changed to “head injury related mortality”.  Such flexibility in trial outcomes breaks one of the golden rules of EBM 2.0 and renders the claimed findings unlikely to be true and this is exacerbated by the change to an arguably irrelevant disease-specific outcome. TXA made no important difference to all-cause mortality with a RR 0·96 (0·89–1·04).

So the real pre-trial hypothesis, and arguably only relevant clinical outcome, was “does TXA improve mortality in patients with TBI” and there was no suggestion of any benefit here. Combining this with the low pre-trial probability of this being true, the post-trial probability of TXA improving outcome in TBI is negligible.

The EBM 2.0 approach warrants that any changed primary outcome should only be considered an “exploratory outcome” worthy of interest only and always requiring further study. Additionally the real claimed benefit is in a pre-specified secondary outcome which is truly exploratory even in the EBM 1.0 world.

The remaining discussion here is therefore conducted for interest, to highlight how principles of EBM 2.0 are used and demonstrate how bayesian analysis could work to show, in this case, a truly fanciful best case scenario for the probability of TXA being beneficial in TBI.

The benefit claimed in the exploratory outcome (changed primary outcome) of head injury related death (18.5% with TXA and 19.8% with placebo; RR 0·94; 95% CI 0·86–1·02) was very small (NNT=77) which the makes this finding less likely to be real (“The EBM Double Whammy”).

However the real benefit claimed in this study was not even in the exploratory, changed primary outcome, it was in an exploratory pre-specified sensitive analysis which excluded patients with GCS=3 or bilateral unreactive pupils (12.5% with TXA and 14.0% with placebo; RR 0.89; 95% CI 0.8-1.0). Here one end of the CI is 1.0 so we can presume the attendant p value would be on or about 0.05.

Perhaps due to realisations about the problems with p values, they have chosen to publish confidence intervals instead. However the use of dichotomous confidence intervals are just as problematic as the use of dichotomous p value thresholds (<0.05) so no advances in the quality of statistical interpretation are made here. Additionally, without p values it makes it difficult to use Bayesian analysis to estimate post test probabilities.

 

Calculating the post-trial probability that the hypothesis is true

So let’s take this presumed p value of 0.05. As described in our article about Converting p Values Into the Probability That a Finding is Real, this p value can be converted into a maximum Bayes factor of 2.44 which acts as a likelihood ratio converting a pre-trial probability of 5% into a post-trial probability of 11%.

However, this was an extremely optimistic estimate because:

• The Bayes Factor used is the maximum possible version and so the actual true hypothesis probability can be far less than this
• This does not take into account bias in the study, both that which is undetectable and that which is detectable, which will further reduce the probability that the findings are real.
• This analysis is not even about the original primary outcome making the findings even less likely to be true (nor is it the outcome we really want to know).
• The benefit in this exploratory outcome was very small making the findings less likely to be true.

Consequently this 11% chance of the hypothesis being true is a fanciful gross overestimate calculated to show demonstrate how Bayesian analysis works using p values.

Finally, without being provided a p value for all-cause mortality, we cannot calculate the maximum post-trial probability but given the CI was above 1, we know the p value was > 0.05 and maximum Bayes Factors for such values are so low that they would barely budge the pre-trial probability of 5%. For example a p value of 0.10 attends a Bayes Factor of only 1.6 which would change a pre-trial probability of 5% to an absolute maximum post-test probability of only 8% (instead of 11%) before discounting this further for the factors explained above.

Even if the suggested 5% pre-test probability was considered too conservative  and a higher figure such as 10% was used, using a Bayes factor of 1.6 only improves this to a post-trial probability of 15%.

SUMMARY: After CRASH 3, the post-trial probability of the original and only relevant hypothesis that TXA improves all cause mortality in blunt TBI injury is essentially the same as the pre-trial probability, which was exceedingly low.   

 

Certainty for Change Threshold

In considering the “real world” cost and benefits of changing practice it is worth nothing that TXA is cheap and in this study and others has demonstrated no definite evidence of harm though concerns have been raised about the potential inaccuracy/incompleteness of the measurement of harm in these studies. On the other hand, TXA is delivered as a rapid infusion then a long 8hr infusion and the latter can arguably significantly complicate and interfere with overall care of patients who are critically unwell. Considering these factors one might consider changing practice at relatively low levels of certainty  (say for example 30-40% certain), compared with the usual very high certainty (e.g. >95%) that we may want for new expensive drugs.

Given the very low pre-trial probability of benefit which is not substantially altered by the results of CRASH 3, it would be reasonable to argue that the threshold of certainty required for practice change has not been met.