Investigative reporters prod at an issue hoping to get a response. David Tuller’s piece on the problems with the PACE trial has definitely done that. Citing “numerous inaccuracies”, “reputational damage” and a social media flareup the study authors are definitely feeling the heat. They demanded equal time on the Virology blog to present their case and they got it.
Whether that’s going to help them is another question. They probably didn’t help their case by stating that Tuller made no attempt to contact them after Tuller reported that he offered to fly to the UK in person to meet with them and his attempts to interact with them via email failed.
So they got off to a bad start. But what about the main issues?
- Did they actually create criteria that produced the possibility that patients could be labeled as disabled and recovered at the same time?
- Did they really in the middle of the trial suddenly and significantly relax the criteria for recovery?
- Did they actually change the criteria for improvement in midstream to those a recently published trial suggested might improve their results?
- Did they pump up the participants in the trial by providing them mid-trial glowing accounts of how helpful it was?
Did they, in short, appear from the outside like panicked researchers desperately trying to salvage a failed trial?
Tuller asserted that the PACE authors attempts to correct his “inaccuracies” lacked substance and skated around the main issues raised. Let’s see what how they dealt with several of the issues Tuller raised.
13% of Patients in the Trial Meet a Threshold for Recovery On Entry into the Trial
How do you get around the fact that a significant percentage of the patients meet one of the thresholds for recovery upon entry into the trial? Lacking the ability to challenge it the PACE authors simply changed the subject and asserted that the patients needed to meet other markers to be labeled as “recovered”.
But really – so what? What about the fact that 13% of the patients – you know the sick people the trial was attempting to get well – were already halfway home to “wellness” (according to the PACE criteria) before the trial even started? How can you trust a trials results when they were to some extent guaranteed? (If 13% were halfway home I’ll bet 25% were knocking on the door; i.e. they just needed a little boost to meet one of the two factors for recovery – before the trial had even started.) Talking about gilding your lily.
If the authors could have gotten around this issue they would have – they’ve been dealing with it for years. They couldn’t….
- Retraction Watch – Score I
PACE Biased Results By Cheering the Participants On
This is another toughie for the PACE crew. Since they couldn’t deny they pumped the participants up mid-trial they could only argue that doing so was OK. That assertion was completely at odds with the study experts Tuller contacted.
Their newsletter did get the OK from their Independent Review Board…. suggesting that maybe things are done differently in the UK than in the U.S. However it happened Tuller asserted they were simply wrong:
“They introduced an uncontrolled, unpredictable co-intervention into their study, and they have no idea what the impact might have been on any of the four arms.”
- Retraction Watch – Score II
Bias Was Introduced by Changing the Two Primary Outcome Measures and How They were Analyzed
The PACE authors simply stated that they found better outcome measures and decided to employ them. Why they were better no one knows. Why the PACE authors didn’t do the sensitivity analyses Tuller believes such changes demand is unclear as well. The only thing we do know is that a similar, recently published trial got better results when they implemented the same changes the PACE trial did.
Hey, if it looks and walks like a duck – it’s probably a duck. This looks from the outside like an attempt for the PACE trial authors to pad their results.
- Retraction Watch – Score III
PACE Authors Revised the Criteria Downward For Most of the Recovery Measures
Again, the PACE authors admitted they relaxed most of the measures of recovery but beyond stating that they felt the new measures better reflected recovery they said nothing more. They didn’t just “relax” the requirements for recovery, however, they practically obliterated some of them. One measure that fell from 85 to 60 left the PACE trial with a “recovery” score of 60 and a “serious disability” score of 65. They also decided later in the trial patients who stated they were “much better” had, probably unbeknownst to them, actually recovered. One wonders how a trial could fail with such a headwind behind it…
- Retraction Watch – Score IV
Check out more issues the PACE authors raised and how Tuller reacted to them in the full blog here.
Tuller’s Full Pace Trial by Error PACE Series
- Trial By Error : the Troubling Case of the PACE Chronic Fatigue Syndrome Study I- the PACE Trial Deconstructed
- Trial By Error : the Troubling Case of the PACE Chronic Fatigue Syndrome Study II – the PACE Trial is Published
- Trial By Error: the Troubling Case of the PACE Chronic Fatigue Syndrome Study III: After Publication
- PACE Authors Respond
- David Tuller Responds
The PACE trial authors appear to have done themselves few favors by presenting their side of the argument.
Their responses demonstrated two things:
- Tuller’s critique and the response to it by the patient community has hit home and
- they don’t have answers to most of his questions.
Notice that every change they made it more possible for the trial to get positive results. Notice how moderate the results were even with these added boosts.
Consider the fact that the researchers were never blinded to the results flooding in. Consider the possibility that they gloamed early on to the fact that things were not going their way and that changes needed to be made.
While the study findings were initially lauded (and misinterpreted) by the press (apparently with Chalder’s help) the controversy over the PACE trial continues to muddy it’s results.
Remember we’re talking about what is surely the most expensive study ever done on ME/CFS. The PACE authors are right. Reputations are at stake. Until they can satisfactorily explain – if that’s possible – the many questions surrounding the trial – it doesn’t appear that the controversy is going to go away.
Troubled by the PACE trial? Sign the petition to compel Lancet and other journals retract numerous claims in the PACE study papers including that 30% of the trial participants recovered.