When bad science becomes orthodoxy

Reflections from a decade of debunking the PACE trial

Today’s guest post is from David Tuller. David Tuller, DrPH, is a senior fellow in public health and journalism at the University of California, Berkeley. Since 2015, his Trial by Error project, on the science site Virology Blog, has investigated scientific, methodological and ethical problems with research in the field of ME/CFS and, more recently, Long Covid. His position at Berkeley is supported by crowdfunded donations to the university, largely from patients and advocates.

---

Back in the 1990s, when I was a newspaper reporter in San Francisco, I never thought to myself, “Hey, I’ve gotta be the guy who writes about ‘chronic fatigue syndrome’ [CFS]!” I mean, why would any reporter seeking to make his or her mark have aspired to that? The dismissive name given to the condition ensured that no one could take it seriously. But then, when a major 2009 paper (now retracted) reported a link between the illness and a mouse retrovirus, I wrote a piece about it for The New York Times. After that I wrote a few more stories—a dozen or so in all. The purported mouse retrovirus ultimately turned out to be a lab contaminant, and media interest in the illness, which had previously been sparse, once again vanished. That drop in news coverage bothered me, as I’d learned in the course of my reporting that the ailment was both devastating and widely misunderstood.

Uncovering the PACE trial

In 2015, I stumbled across a medical scandal that no journalists were covering—the saga of the PACE trial. In this £5-million study, which was funded by UK taxpayers, the investigators examined whether their two preferred treatments reported to be harmful by patient groups—an incremental increase in activity called graded exercise therapy (GET), and a specialised form of cognitive behaviour therapy (CBT)—were effective for what is now known as myalgic encephalomyelitis (ME), or by a hybrid term, ME/CFS. The Lancet, one of the world’s leading journals, published the initial PACE results. The lead investigators were well-known British mental health experts, luminaries in the medical and academic firmament. They had long contended that physical deconditioning was causing symptoms in patients because they had been sedentary during their recent illness. And patients remained sedentary because they mistakenly believed they were suffering from an underlying organic disorder made worse by minimal exertion.

The goal of GET was to counteract the deconditioning and get patients “reconditioned.” The goal of the CBT was to help patients let go of their false illness beliefs. The Lancet paper, and subsequent results published in other journals, purported to prove that the two interventions could successfully treat the illness. The findings were widely embraced, not just in the UK but around the world. PACE was routinely cited to reaffirm GET and CBT as the treatments of choice.

What went wrong with PACE?

As patients already knew, PACE was a methodological and ethical disaster—a piece of crap. It was an unblinded study that relied solely on subjective outcomes for its reports of success. That study design, while common for investigating non-pharmacological interventions, is a recipe for generating bias. When you give one group of people a course of therapeutic treatment that they believe might help them, and ask them afterwards how they feel, they are more likely to say positive things than patients who did not receive the treatment.

In the PACE trial, that’s exactly what happened—those who received GET and CBT reported modestly better results for fatigue and physical function than those in the comparison arms. By contrast, the study’s objective measures—such as a six-minute walking test, a step-test for fitness, employment status, and whether they were receiving social welfare—showed no meaningful improvement. In the face of this embarrassing failure, the PACE investigators rejected their own objective outcomes as inadequate and essentially meaningless.

The study contained other indefensible flaws. Long after data collection began, the investigators dramatically weakened the scoring thresholds for “recovery” on their two primary outcomes—self-reported fatigue and self-reported physical function. In fact, the new outcome thresholds for “recovery” were even lower than the scores required on the same measures to be “disabled” enough to enter the study. As a result, 13% of the study participants were simultaneously “recovered” and “disabled” for physical function when they entered the study—a logical impossibility. In a flagrant example of research misconduct, the investigators did not disclose this highly salient detail in their papers.

Moreover, the main investigators all had extensive ties with insurance companies seeking to limit health and disability claims. These ties were acknowledged in published papers but not disclosed to study participants, although the approved trial protocol mandated such disclosures. Even as the study was documenting that the treatments did not help people get back to work, PACE investigators were telling insurance companies the opposite.

Shifting the orthodoxy

In October 2015, Virology Blog posted my 15,000-word investigation of the PACE mess, called “Trial By Error: The Troubling Case of the PACE Chronic Fatigue Syndrome Study.” I expected the effort to be a one-off, but I kept finding more angles to expose. At first this work was a public service effort, unrelated to my academic position at the University of California, Berkeley, running a joint master’s program in public health and journalism. Since 2017, however, the Trial By Error project has been my main academic focus. The costs of my position are supported by donations, largely from patients, carers and advocates. (I do not accept donations personally; they are made directly to Berkeley through the in-house crowdfunding platform.)

Since this project began, I have written hundreds of blog posts as well as peer-reviewed papers, opinion pieces and commentaries for news organisations, and pointed letters to editors and journals about sub-par research in this field. A 2018 open letter I organised and sent to The Lancet called for an independent investigation of the PACE trial and was signed by more than 100 experts from around the world. The letter received high-profile news coverage in both The Times and The BMJ. I considered that a real coup.

Credit: Invest in ME Research, 2017. David Tuller presenting his work on the PACE trial at the 12th Invest in ME Research International ME Conference in London. Skip to 35 minutes to watch David tear up a copy of the paper and toss the pieces in the air.

In 2021, the National Institute for Health and Care Excellence (NICE) published its new ME/CFS guidance, which rescinded the agency’s previous recommendations for GET and CBT. (I had organised an open letter to NICE, signed by more than 150 experts and many dozens of patient and advocacy groups, to support the release of the proposed guidance in the face of howls of protest from leading GET/CBT ideologues.) In assessing the research, NICE had found that the quality of the evidence in favour of both interventions, including from the PACE trial, was of poor quality.

Helping to achieve such an outcome had been one of my goals, so I considered the change to be a significant development. I figured it might be time to sign myself off, call it a day for Trial By Error, and pursue some other academic and journalistic project.

But Long Covid was already upon us.

From ME to Long Covid

The fight over the 2021 NICE guidance for ME/CFS was particularly fierce because everyone involved undoubtedly recognised that it was a proxy fight over how to view and treat Long Covid—or, more precisely, that subgroup of Long Covid patients who meet criteria for a diagnosis of ME or ME/CFS.

As soon as the pandemic started, everyone in ME-world knew what would happen: after an acute bout of illness, a significant minority of people would report prolonged, non-specific symptoms. Despite emerging biomedical evidence of immunological, neurological and other dysfunctions in Long Covid patients, they would be told they were suffering from anxiety, depression and related disorders—and that’s what we’ve seen.

Members of the GET/CBT cabal have continued to parrot their theories about deconditioning and “unhelpful beliefs” and have transferred that template wholesale to Long Covid. And they continue to cite the PACE trial favourably in this context—even though its findings have been rejected by NICE, the US Centers for Disease Control and Prevention, and other healthcare organisations around the world.

So here we are. The paradigm embodied by the PACE trial has been effectively debunked—and yet it refuses to disappear. So, I plan to keep on slogging away for at least a while longer, seeking to push back against the scientific garbage that continues to pollute the medical literature in this field. Other projects will have to wait.