“…many of these people spout nonsense and then freak out when confronted with facts or details about the research that they can’t explain away.” David Tuller
David Tuller is a unique figure in the ME/CFS universe. A reporter and editor for the San Francisco Chronicle, Salon and writer for the New York Times, the policy journal Health Affairs, and other publications, Tuller became outraged at the long history of neglect and abuse directed toward ME/CFS.
His publications date back to 2011, when he published the long exposé, “Chronic Fatigue Syndrome and the CDC: A Long, Tangled Tale”, on the history of the CDC’s CFS program under Dr. Bill Reeves.
It was the largest (n=640) and most expensive ($8 million) clinical trial ever attempted in chronic fatigue syndrome (ME/CFS) that really got Tuller rolling. Intended to be the trial to end all trials in ME/CFS, the trial proponents hoped it would show that cognitive behavior therapy (CBT) and graded exercise therapy (GET) could get ME/CFS patients out of their doldrums and back to being productive members of society – and in a cost-effective manner – to boot.
With its $8 million price tag and hundreds of patients, the stakes were high. Lead by Peter White, Michael Sharpe and Trudie Chalder, the PACE trial featured the cream of the crop of U.K. biopsychosocial researchers, who, perhaps worried that the trial would not succeed, made a series of controversial changes to the trial protocol that ended up significantly bolstering its results.
While the results were never that great, the trial seemed to be a hit. Bolstered by the researchers’ comments to the media, the major news organizations lapped up the good news like honey: CBT/GET can help or even cure ME/CFS.
- “The Medical News Today” reported that “Fear of exercise is biggest barrier to chronic fatigue syndrome recovery“.
- “Psychotherapy Eases Chronic Fatigue Syndrome, Study Finds”— New York Times
- “Pushing limits can help chronic fatigue patients”— Reuters
- “Brain and body training treats ME, UK study says”— BBC
- “Therapy, Exercise Help Chronic Fatigue Syndrome”— WebMD
- “Helping chronic fatigue patients over fears eases symptoms”— Fox News
- “Chronic fatigue syndrome patients’ fear of exercise can hinder treatment – study”— The Guardian
- “Chronic Fatigue Treatments Lead To Recovery In Trial”— Medical News
Critiques of the trial emerged quickly from patients and patient organizations, but to no avail. Since 2011, the study, which the Lancet posted as an open access article – thus ensuring it would get maximum coverage – has been cited by other journals 675 times, and ended up playing a major role in the UK’s National Health Service, the Centers for Disease Control, the Mayo Clinic, the 2013 Dutch, and Kaiser‘s guidelines for using CBT/GET in ME/CFS. One reporter stated that, “PACE has become the paradigm for understanding a condition affecting millions of people.”
David Tuller’s 15,000 word 2015 investigative piece, “TRIAL BY ERROR: The Troubling Case of the PACE Chronic Fatigue Syndrome Study“, published on the Virology blog website, galvanized efforts to expose the trial’s shoddy methodology. Arguing that trial results were built on a house of cards, the article played perhaps the key role in lighting a fire which is still burning today.
Eight years later, the trial is shrouded in controversy. White, citing stress, has retired, Sharpe is no longer studying ME/CFS, and statisticians have trashed the Lancet for: a) publishing such a shoddy piece of work; and b) not retracting it. One hundred and fourteen academics and experts signed on to letters denouncing the methodology used in the trial. A Columbia statistician, Professor Bruce Levin, used the PACE trial as the case study in a talk last summer titled, “How NOT to Conduct a Randomized Clinical Trial,” and called it “the height of clinical trial amateurism.”
After Queen Mary College of London University’s attempt to block the release of the raw PACE data proved to be an embarrassing failure, a re-analysis of the trial, using the original protocol, demonstrated what many suspected – that the trial was largely a failure. Remarkably, every modification to the trial protocol had served to boost its results – in one case by 400%.
Tuller’s disgust with the PACE trial launched him on a unique, patient-supported investigative effort lasting two years, which has grown to focus on ferreting out the bad science permeating the biopsychosocial field. Tuller’s analyses of the Cochrane Report on GET in ME/CFS, the School Absence study, the Lightning Process study, and others have sparked questions and in some cases investigations.
Now, as Tuller raises money for a third year of investigative journalism, it seemed like a good time to catch up with him and his work.
David Tuller Talks
From my perspective, debunking bad research like PACE is the most effective way that I personally can improve health outcomes on a population level. David Tuller
You reported that the BMJ published an article calling you an “activist” while neglecting to include your professional credentials – you have a Doctor of Public Health degree. (The credentials of the pro-PACE commenters were all included.) That seemed intended to negate or slight your contributions. What is a Doctor of Public Health degree?
Good question! It’s a DrPH. I’d never heard of this degree before I became a public health masters student at Berkeley. In the US, many disciplines have both academic doctoral degrees — i.e., a PhD — or so called “professional” doctoral degrees. So in public health, a DrPH degree is considered interdisciplinary and geared toward problem-solving or translating research into practice.
In my case, I received my DrPH while already on the journalism faculty. I was able to continue to take courses and pursue the research for the degree as an employee benefit. I’ve never held myself out as being anything other than a social scientist in the public health field. I’m obviously not a medical doctor or a biologist or an epidemiologist or biostatistician. It’s strange to sort of be criticized for not being what I’ve never claimed to be, which is what seemed to happen in the recent Reuters piece.
With regards to the piece in the BMJ — I don’t think the writer realized, so I don’t think it was intentional. I’ve never had any contact with the journalist. I think it’s actually a bit more pernicious. He just assumed that anyone opposed to PACE was an “activist” by definition and didn’t have credentials. Fiona Godlee knows very well who I am, and so the piece was corrected pretty quickly. Of course, they added my credentials but did not remove the word “activist,” which pissed me off. I would guess the reason why is that adding something is a “clarification” while removing something is automatically a “correction.” But they have provided no explanation for why I am an “activist” for advocating my professional and academic view about this research, while those defending it are presented as just doing their academic thing.
How have the PACE trial authors responded to the many critiques of the PACE trial?
Well, yeah, they’ve “responded” many times, as they keep whining. The problem is that they keep providing non-answers. They keep repeating the same stupid things. And that essentially boils down to, ”We made changes and we think the changes we made are good, so there!” That worked for quite a few years, apparently. It’s not working too well for them anymore. The narrative has changed. They now have to portray not only patients as hysterical and oppositional, but me, my colleagues at Berkeley who use PACE as a case study of bad research, and the 100+ experts from around the world who signed that last open letter to The Lancet. It’s gotten to the point that the only way Reuters, for example, could present the side of the CBT/GET ideological brigades is to pretend only me, patients and Dr. Myhill in the UK think PACE stinks.
The Lancet PACE study was just the beginning of a series of studies that were published on the PACE trial. If the Lancet PACE trial falls, will the rest of them go as well?
Yes and no. That’s complicated. They all suffer from being open-label studies relying on subjective outcomes. That should be a non-starter in terms of relying on such studies for policy-making, but unfortunately that is not the case. PACE is also likely research misconduct on top of bad science, because of all the data manipulations and the decision to not tell readers that, for example, 13% of the participants had met the outcome thresholds for “physical function” at baseline. You can’t hide stuff like that in research. It’s wrong.
The other issue is that many of these are Oxford criteria studies, so we don’t even know who is really in the sample. Certainly, if PACE were to finally be officially recognized as garbage, that would have a big impact. But my predictions aren’t really that good — four years ago, I thought if I wrote a 15,000-word investigation and revealed what I did, the trial would not survive. Certainly if this were a trial of any other illness that people believed existed, it would have been retracted long ago, or never published.
The PACE trial has been bashed all over the place, but Richard Horton, the editor of Lancet, still seems unresponsive. What are you and others requesting that Horton do, and if he continues to be unresponsive, what, if anything, is there left to do to get him to change his mind?
I originally assumed people were thinking clearly, and that if you point out that people could be “within normal range” for physical function at baseline, and then you also point out that 13% of participants were in fact “within normal range” for physical function at baseline, there would be a response. Horton does not seem to think there is a problem with the study. I mean, among all the others. He will never change his mind. He came out swinging when patients first began criticizing PACE, and his reputation is tied to it, I’m afraid. Or at least maybe that’s how he sees it. I don’t know. It’s very bizarre. Then he runs all over the place talking about the problems with research. I want to say to him, clean up your own back yard first.
What I don’t get is what Lancet gets out of this? Having over 100 academics say a trial published in your journal is trash isn’t good, but having the American Statistical Association urge its conference participants to attend a talk on PACE to learn “how bad statistics harm patients and our profession” is beyond embarrassing.
That’s just the beginning. The Journal of Health Psychology devoted an entire issue to the PACE trial. Rebecca Goldin, a Professor of Mathematical Sciences at George Mason University, stated that the shoddy way the PACE trial and its aftermath have been handled at the Lancet (which pushed it through in 4 weeks) had “sent shockwaves through medicine.” Professor Bruce Levin has stated that the Lancet’s defense of the PACE trial had diminished the respect The Lancet is held in “worldwide”.
Do you have any idea why Richard Horton, the longtime editor of the journal, who, by the way, has taken the medical field to task for its often “poor methods” and misleading results and portrayed himself as a straight shooter, is willing to expose his beloved Lancet to such ridicule? At times he seems almost beside himself with anger at the “activists” who have raised questions about the trial.
I really don’t get it. I also don’t get why The Lancet fast-tracked the study to publication. My assumption, and I have no way of knowing if any of this is the case, is that he was assured it was a great trial and took these people at their word. And now he’s kind of gotten himself stuck in a place he can’t get out of. He’s defended it so many times, he’d look like an idiot to back down. But he looks like an idiot anyway because he’s defending something that objective observers know is ridiculous. Poor guy!
You’ve also criticized the Psychological Medicine PACE recovery study, where, as you noted, “the investigators weakened all four of the ’recovery‘ criteria laid out in their protocol, because, as they have explained, they determined that their initial assessment methods were too ‘stringent’ to demonstrate ‘recovery’. In other words, they were concerned that their reported ‘recovery’ rates would be too low.” The recovery criteria were so lax that 13% of the ME/CFS patients who entered the trial were already deemed recovered. What is the status of the Psychological Medicine study?
The status is the same. It’s in the literature, and so people can cite it. That study really should be retracted immediately. The single analysis is completely bogus because they weakened all the criteria, because of the overlap you mentioned, and because they do not appear to have received any oversight committee approval for these dramatic changes in the recovery definition. No study should be published with these egregious issues. The editors have behaved disgracefully — but then Sharpe is or was involved with the journal. So a lot of this, I assume, is just how they’ve always done things.
Peter White, the senior author of the PACE trial, has retired, Simon Wessely is long gone, Michael Sharpe is no longer working on ME/CFS, and the Reuters article indicated that others have left the field. A Dutch Health Council recommended against using graded exercise therapy (GET) and focusing more on biological research. The Cochrane Reviews, at least temporarily, has pulled an analysis of ME/CFS GET studies over concerns of accuracy. After the AHRQ in the US pulled its recommendations for CBT/GET, so did the Centers for Disease Control. Now NICE is reviewing its guidelines for CBT/GET. Is the biopsychosocial approach to ME/CFS, which has consumed so much UK and European funding, in decline?
It certainly looks that way from the perspective of the US, especially when it comes to ME/CFS. And that’s great. At the NIH meeting in early April, there was not an iota of skepticism expressed that this was anything other than a devastating illness and that the answer was biomedical. That’s putting aside the question of whether specific research was good or bad, or whether NIH needs to ramp up spending dramatically. On the scientific level, everyone has moved beyond that framework. Just to be clear — it will take longer for that change in attitude to spread among health care providers, so people might not feel that change immediately in their own medical care, unfortunately.
And some of these issues aren’t settled at all in Europe. We don’t know what Cochrane will ultimately do with the exercise review. We don’t know what the NICE guidance will say. In Finland, Norway, Denmark, Germany, etc., the CBT/GET ideological brigades are very strong. In the UK, they might lose some ground on ME/CFS but make it up by treating people with IBS or other illnesses falling under the category of “medically unexplained symptoms”, or MUS. Especially because they are expanding the program called Improving Access to Psychological Therapies, or IAPT — an NHS program designed to shunt people with many illnesses away from medical care and into short-term CBT and related therapies.
The National Institute for Health and Care Excellence (NICE) in the UK will publish a review of its guidelines for CBT/GET in ME/CFS next year. Do you have hope that any substantial changes will occur?
Well, I have more optimism than many in the UK who have been down this road so many times and faced such nonsense for decades. There are some smart people on the panel who have their heads on right, and then there are some biopsychosocial folks, and more of them than would appear warranted. And there are huge interests at play, and lots of money and lots of embarrassment to go around the entire UK academic and medical establishment around this whole issue. So, in some ways, how that plays out is unpredictable.
But the reason I have some optimism is that many of these people spout nonsense and then freak out when confronted with facts or details about the research that they can’t explain away. So, these folks on the panel who will be promoting CBT and GET will have to come up with some decent answers to explain their reasoning. Those answers don’t exist. Now, of course, I could be wrong, and if it comes down to a strict vote, who knows what will happen? But there are people on the panel who will be arguing very hard for what is right, and with all the facts on their side — and that’s a huge difference from the last time around.
Do you believe Dr. Sharpe and others when they say that they believe that chronic fatigue syndrome is a biological condition?
That is really sophistry. Everything is biological. Deconditioning is “biological.” I believe they believe people have real symptoms due to deconditioning. When they say they believe in “biology”, they often mean they acknowledge that it is triggered often by an infectious illness. So, in that sense, they have always believed in “biology”.
They have publicly insisted, however, that the perpetuation of the symptoms is due to deconditioning, which is due to unhelpful illness beliefs. Now, I have no idea if at this point they actually still believe that. I would actually assume no and they’re trying to massage how they talk now. However, I have never claimed that they believe it’s “all in the head” because that’s not true. They seem to believe the symptoms are very real, as I read their work. In their theory, the symptoms are the symptoms of deconditioning and accompanying disruption of circadian rhythm, muscle atrophy, etc.
What effect has the CDC’s decision to drop CBT/GET from its guidelines and pull any references to the PACE trial had?
It will ultimately have a huge impact. It’s the CDC. If they don’t believe the science holds up, people pay attention to that. But if the CDC actually behaved like a public agency, this change would already have had a much bigger impact. As soon as they dropped those from the recommendations, Dr. Unger or someone from CDC should have gone out and made a statement about it and informed Kaiser Permanente, Mayo Clinic, and so on. Even just put out an advisory or press release! Something. But they did nothing. I noticed that it had happened because someone mentioned it to me, and so it only became public when I asked the CDC about it.
Then they started making the ridiculous claim that they removed the recommendations because people misunderstood or took then the wrong way. That is really nonsense. It’s face-saving. And I find it very dispiriting to see public health professionals acting that way — since that’s what I am. As public health officials, they have an ethical and professional obligation to acknowledge that they got it wrong, that it was a mistake to be citing PACE, that everyone understood what they meant by CBT and GET, and that they are sorry about the whole mess. Of course, they haven’t done that.
In the past year you traveled to Australia, the UK and Ireland and, of course, you’re from the US. Did you find differences or similarities in how ME/CFS is viewed, studied and treated in these countries?
Well, the most striking difference is in nomenclature. In Australia, patients, clinicians, researchers all say “chronic fatigue” to mean ME, as used by people in the UK. When I was there, at first I corrected people, and then I just gave up. When I gave talks, I would say that you can’t blame others for conflating the illness and the symptom of chronic fatigue if you’re going around calling it “chronic fatigue”. And people would nod and say yes, yes, and then they’d refer again to “chronic fatigue.” So, I decided I’d either tear my hair out or just not let it bother me.
In the US, of course, no one has heard of ME. If you mention ME in the UK, people not involved say, oh, chronic fatigue syndrome, right? Or at least that’s been my impression. In the US, zero recognition of the term ME. Now, of course, in the US, the government agencies are calling it ME/CFS, but most people have no idea. So, you can’t really get around yet from referring to it often as chronic fatigue syndrome. It’s challenging to write about also because the studies all use varying terms. If a study calls itself a study of whatever in CFS, I kind of try to use CFS when writing about it.
Tuller on his Crowdfunding Effort
What’s on the tap for next year? What do you hope to see happen?
I’d like to see enough happen that maybe a year from now, I can think about, you know, something else! I’m joking, but serious also. It would be nice to feel that there’s been enough momentum and that it feels real. I still really find working on this project rewarding and fulfilling. I do feel that I need to focus more on the MUS steam engine as it picks up speed, and look more at other complex illnesses that fall under the MUS umbrella.
In the US, I’d like to focus on how care is being delivered, what care are people getting, what are the major medical centers doing now that Kaiser Permanente has at least made steps in a positive direction. I think it’s also important to keep up pressure on the NIH and the CDC to do their jobs, and that includes ensuring that case definitions are used appropriately.
I have training in public health, so while the science is very interesting to me, I tend to focus more on the impacts of things in the world. Public health is really about considering health on a population level. From my perspective, debunking bad research like PACE is the most effective way that I personally can improve health outcomes on a population level. So this is both my public health project, and it’s also my investigative journalism project. It’s kind of a perfect project for me in that way.
Help fund David Tuller in his work over the next year here. Your contributions are tax-deductible