Solve ME/CFS took its shot at the AHRQ report and it didn’t pull any punches. The Solve ME/CFS review is notable because it was produced by an experienced researcher, Suzanne Vernon PhD, in collaboration with a Research Advisory Board that is made up of doctors, researchers, and industry leaders.

(In contrast, the Health Rising report was done a by layman (me) and reviewed by Jim – Health Rising’s editor :)).

When you have this many PhDs and MDs (see list below) weighing in on a federal report on Chronic Fatigue Syndrome, it’s good to hear what they have to say.

Solve ME/CFS Research Advisory Board

• Suzanne Vernon, PhD, Research Director of the Solve ME/CFS Initiative
• Italo Biaggioni, MD, Professor of Medicine, Professor of Pharmacology, Vanderbilt University
• Gordon Broderick, PhD, Professor, Center for Psychological Studies, Nova Southeastern University
• Russell “Rusty” Bromley, Principal, TRAC Consulting
• Paul R. DeStefano, JD, Partner, McDermott Will & Emery LLP
• Roger Y. Dodd, PhD, Vice President, Research and Development and Director, American Red Cross Holland Laboratory
• Mark A. Demitrack, MD, Vice President and Chief Medical Officer, Neuronetics, Inc.
• Kimberly McCleary, Patient-Centered Research Advocate and Consultant
• Kelly Morren, BS, Global Director, ServiceSource International
• Bernard Munos, Principal, InnoThink
• Peter C. Rowe, MD, Professor of Pediatrics, Johns Hopkins Children’s Center
• Tomasz Sablinski, MD, PhD, Founder and CEO, Transparency Life Sciences; Managing Director, Auven Therapeutics
• Terry Tyler, MD, Diagnostic Radiologist

The Solve ME/CFS overview of the AHRQ report comes to two main conclusions:

• The P2P process is not going to be effective in informing healthcare decision-making. Because of the “comparative effectiveness methodology” it used – which the Chronic Fatigue Syndrome research field is weak in – the vast majority of ME/CFS studies were excluded from analysis. Few recommendations on ME/CFS diagnosis and treatment are going to ensue from the report.
• The report will, however, highlight basic research elements that are needed to move this field forward.

How to Blow Almost 200 Million Dollars

“Not only has the NIH egregiously underfunded this illness (based on prevalence), but the research NIH has funded has not contributed to an evidence base that has moved the science forward.”

Solve ME/CFS  noted that since 1991 the NIH has spent 191.5 million dollars on ME/CFS, yet the AHRQ ended up rejecting ninety-nine percent of the NIH-funded studies because of design flaws.

It’s true that these studies opened new ground and new insights into Chronic Fatigue Syndrome; the money is not wasted.  But because the studies also  failed to incorporate the design elements needed to validate them, it wasn’t well-spent either.  It all adds up to the fact that twenty-five years and hundreds of millions of dollars later we still have no biomarkers.

Essentially what we have now are lots of good leads. What we now need are the kinds of studies that can validate them.   

The P2P systematic review of the ME/CFS evidence base illustrates the lack of rigor in crucial research design elements and the absence of high quality clinical trial data.

[Consider that if the design elements the AHRQ deems necessary had been included, the report could have stated something like, “ME/CFS is a disease characterized by natural killer cell dysfunction and reduced aerobic energy production during exercise.” THAT would have been a game-changer.]

Solve ME/CFS noted that multiple sclerosis, a disorder with validated diagnostic biomarkers, has received $4 billion in funding over the past 25 years while ME/CFS has received 200 million.  That’s twenty times as much funding for MS.

Compare that $4 billion to the $400 million spent on Fibromyalgia, which is also at the bottom of the totem pole at the NIH and, in fact, receives less funding per patient than ME/CFS. Neither ME/CFS nor FM have validated biomarkers.

Clearly, neither researchers nor the NIH grant panel members who approved the ME/CFS grants were aware of all the parameters needed. The Solve ME/CFS report took the NIH to task for both underfunding  this disorder and not funding the types of studies needed to move this field forward. They also called on them to ensure that right kind of studies are done in the future.

No Surprise!

While patients and some researchers were shocked at the reports findings, others were not.The problems facing the ME/CFS research field have been known for quite some time.

Solve ME/CFS noted that the 2011 State of the Knowledge Symposium highlighted problems with study design, definitions, and lack of biomarkers. In 2012 the FDA stated that the lack of an accepted ME/CFS definition, or accepted outcome measures, or biomarkers providing simple, quantitative ways to assess disease activity were keeping drug companies away. In 2013 the FDA highlighted the need for the field to utilize “comparative effectiveness methodologies”- the same criteria cited in the AHRQ report.

No Bias in the Report

The ME/CFS community is often understandably worried about bias, but SolveME/CFS reported the AHRQ panel members simply systematically applied a standard set of research design criteria known as PICOTS (Populations, Interventions, Comparators, Outcomes, Timing, and Setting) to ME/CFS studies.

The Good News

“The P2P process may be what is needed to break through that negative cycle and move us to a day when ME/CFS is appropriately funded.”

The good news is that the report does lay out design elements future studies should include.  The other good news is that the very existence of the IOM and P2P reports suggest the federal government is starting to pay attention to ME/CFS.  Hopefully both reports will call for increased funding to help clear up the barriers to  progress in this field including the need for a validated definition and diagnostic biomarkers.

See the full Solve ME/CFS report below the editorial.

Editorial

This report has shown us that even our best researchers with million dollar NIH grants can be caught short. The word needs to get out to everyone before we go another ten years without a single validated biomarker.

What can patients do? They can request that the P2P panel clearly and explicitly address both the gaps in the field and the NIH intransigence in addressing them.

They can demand the NIH produce grant opportunities (RFA’s) aimed at producing a definition and validated diagnostic biomarkers.

They can demand that the NIH’s program announcements require certain design elements be present in every diagnostic biomarker grant application.

They can ask if we’re going to see an organized effort to make sure that the field applies the AHRQ findings to future research.

I’ve had this disorder for 35 years. I’m now 55 years old. I don’t want to be reading the same report when I’m 65.  Reading about the shortcomings of the ME/CFS research field is getting old.

There’s no doubt that pulling a field up by its bootstraps is no easy task, but the AHRQ report indicates it’s a necessary one. Several years ago efforts were begun to standardize the field, but they don’t seem to have  paid off.  It’s too bad – really tragic – that there’s not some ME/CFS study design czar that researchers can go to in order to check out their study designs.

The logical entity to get the study design message out to the ME/CFS research field  is the professional association of ME/CFS, the IACFS/ME.  An IACFS/ME report outlining study design needs to be prominently posted on their website, and sending it to their members would be helpful.

Patients who donate to research efforts can ask if the reports findings will be implemented  in future studies. Solve ME/CFS will be providing another grant opportunity next year. Will it require its grant recipients to incorporate the needed study design elements into their projects, if not at the pilot data level, then at the NIH grant level? (Note that projects focused on producing pilot data for NIH grants don’t need to go to these extra lengths, but NIH grant applications based on them should.)  Can the Solve ME/CFS BioBank leverage its resources to ask BioBank users to do the same?

What about the Chronic Fatigue Initiative? Will their diagnostic biomarker studies follow the AHRQ’s lead, or will  they, like 99% of other studies, focus purely on “etiology”?

The primary problems facing the ME/CFS research field with regard to these recommendations are researcher ignorance and added expense.   Researchers may very well balk at adding these extra elements. The researchers’ primary need – to remain employed – simply requires that they publish, not that they have premier study designs.  The AHRQ report shows, however, what a high cost patients pay – 25 years of research with no validated biomarkers – when researchers don’t employ the most rigorous and comparable study designs possible.  That’s a cost patients should no longer have to bear.

The NIH has the money to fund projects like this, but many private efforts probably will not.  That is why the NIH MUST lead in this area with increased funding opportunities for ME/CFS – including the first RFA grant in more ten years.

The Solve ME/CFS Report

Can a Process that is Inappropriate for ME/CFS Inform the Research Path Forward?

Summary Overview

As a research focused organization, the Solve ME/CFS Initiative (SMCI) understands the impact a program like the Pathway to Prevention workshop (P2P) can have on the research landscape.  Because of its importance, we have utilized the collective brainpower of our Research Advisory Council, led by our scientific director, Suzanne D Vernon, PhD, to perform a careful review and response to the Evidence-Based Practice Centers’ draft evidence-based review for ME/CFS. (Read our official submission HERE)  We have concluded that:

• It confirms what we and others have suspected: the Pathway to Prevention (P2P) process is not appropriate for ME/CFS.  Among other reasons, it uses comparative effectiveness methodology to review the evidence to inform healthcare; in the case of ME/CFS the evidence base is much too slim for this method to be effective.
• The P2P systematic review of the ME/CFS evidence base illustrates the lack of rigor in crucial research design elements and the absence of high quality clinical trial data. The review shines a bright light on the need for well-designed, adequately powered studies to identify diagnostic gold standards and safe and effective treatments.

Furthermore, it is important to note that the vast majority of ME/CFS research studies were omitted from this evidence review due to their inability to meet the rigorous standards required for inclusion. Of all of ME/CFS research studies funded by the NIH since 1991 ($191.5 million spent), less than 1% of those studies were included. So, not only has the NIH egregiously underfunded this illness (based on prevalence), but the research NIH has funded has not contributed to an evidence base that has moved the science forward.

This review substantiates the negative cycle currently in play with ME/CFS research – Because there is little funding, there is little research. And because there is little research, there is little evidence to prompt additional funding. But the P2P process may be what is needed to break through that negative cycle and move us to a day when ME/CFS is appropriately funded. It’s existence demonstrates that there is increasing recognition of the clinically devastating nature of ME/CFS and recognition of the disease as an urgent area of future study.

Background – How did we get here?

To review what has led up to this NIH P2P workshop for ME/CFS:

• The 2011 NIH State of the Knowledge Workshop for ME/CFSconcluded that there were problems with definitions, gaps in study design, and a lack of studies on co-morbid conditions, biomarkers, or genetics.
• A year later, at the CFS Advisory Committeemeeting, Dr. Sandra Kweder, FDA Deputy Director of the Office of New Drugs tells us there are so few applications to the FDA because there is no accepted ME/CFS definition, no accepted method for measuring how patients feel or function and no accepted biomarker to provide a simple, quantitative measure of disease presence or activity.
• In 2013, ME/CFS is the first “patient-focused drug development” workshop held by the FDA. The outcome of that meeting, as expressed in the Voice of the Patientreport, is that ME/CFS is a serious disease with significant unmet medical need that requires comparative effectiveness research to generate the evidence to assess the effectiveness, benefits, and harms of different treatment options.
• In 2014, Dr. Susan Maier, Deputy Director of the Office of Research on Women’s Health at the NIH, who oversees ME/CFS research at NIH, successfully garnered a Pathways to Prevention workshop titled, “Advancing the Research on Myalgic Encephalomyelitis/Chronic Fatigue Syndrome”.

For much of this year, the Solve ME/CFS Initiative (SMCI), as well as many other individual advocates and patient groups, has been talking about the National Institutes of Health (NIH) Pathways to Prevention (P2P) for ME/CFS workshop. In April, we first reported on the possibility of the P2P workshop, then in the approval process. Once it was approved, the NIH worked with the Agency for Healthcare Research and Quality (AHRQ) and their Evidence-Based Practice Centers to conduct this evidence-based review.

In June, SMCI expressed concerns that the search criteria used for the evidence-based literature review would bias the review toward studies on CBT and GET. On Sept. 22^ndAHRQ published the draft systematic evidence review on the Diagnosis and Treatment of ME/CFS for comment.  Several members of our Research Advisory Council together with Suzanne D. Vernon, PhD reviewed the draft and have provided peer review comment directly to AHRQ. SMCI and our research advisors believe that this systematic review has identified important and significant gaps in research that have contributed to the dearth of evidence needed to diagnosis and treat ME/CFS. Indeed, it is almost all “gap” and almost no substance.

Dearth of Evidence

From 1991 to 2014, NIH Research Portfolio Online Reporting Tools (RePORT) indicates a total of $191.5 million has been awarded to investigators to directly study ME/CFS or conduct research relevant to ME/CFS.  Spending for other medically unexplained disorders that often occur with ME/CFS, like Fibromyalgia and irritable bowel syndrome (IBS), is about $400 million for each in the same time frame. This is in contrast to multiple sclerosis (MS) – a condition with objective diagnostic biomarkers and treatments – where NIH spending in the same period exceeds $4 billion.

Further, a search of PubMed.gov for anything published on ME/CFS finds 5,450 peer reviewed scientific articles; there are 8,000 articles for Fibromyalgia and 9,000 for IBS.  This is in contrast to the 55,500 articles found in PubMed that make up the MS evidence-base.  There is a direct correlation between funding levels and the breadth of the published evidence-base.

The One Percent

Now, the NIH P2P draft systematic review is out. All told there were 5,901 papers considered by the Evidence-Based Practice Center in the comparative effectiveness systematic review. In the end, less than 1% of these 5,901 paper met criteria for inclusion in the systematic review process. Why? 

• Because comparative effectiveness systematic reviews are based on evidence-based facts. The P2P uses a rigorous process to empirically identify methodological and scientific weaknesses in each published paper.
• The investigators conducting these evidence-based reviews are experts in this review process and come to the task at hand without any bias or preconceived notion about the topic they are reviewing.
• The procedures for these reviews are possible because there are standard methods and procedures in place for conducting research; especially human research intended to impact clinical care. They systematically reviewed the ME/CFS papers using specific research design criteria known as PICOTS – Populations, Interventions,Comparators, Outcomes, Timing, and S

While the draft comparative effectiveness review on the Diagnosis and Treatment of Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) may be a hard pill to swallow for patients and for the many ME/CFS scientists whose research studies were excluded from this review, it identifies vast research gaps and provides important information needed to chart the way forward.

The rationale for evidence-based reports makes sense:

• “Systematic reviews are the building blocks underlying evidence-based practice; they focus attention on the strength and limits of evidence from research studies about the effectiveness and safety of a clinical intervention. In the context of developing recommendations for practice, systematic reviews can help clarify whether assertions about the value of the intervention are based on strong evidence from clinical studies.”

And …

• “This review is not intended to address the question of etiology nor underlying factors that lead to the onset or perpetuation of ME/CFS but rather to focus on the diagnosis and treatment of this syndrome.”

With this is mind, the review identified only 64 studies that met evidence-based criteria to address the key questions; 1) ME/CFS diagnosis and 2) ME/CFS treatment. For a diagnosis study to be included it had to discriminate ME/CFS patients from healthy controls and (ideally) from other diseases with similar symptoms and if the study used biochemical marker it had to be in the context of treatment versus etiology. Twenty-eight studies addressed various aspects of diagnosis. Most of the studies pertaining to diagnosis were rated as fair quality (on a scale of “good”, “fair”, “poor”).

These diagnosis studies showed that certain measures that assess function are able to distinguish between ME/CFS and healthy people reasonably well. The studies were rated “fair” because they were small and diseases similar to ME/CFS were seldom included. The lack of a gold standard for diagnosis of ME/CFS limits how broadly the limited evidence base can be generalized to the ME/CFS community as a whole. The review notes that clinical experts identify post-exertional malaise (PEM) as a cardinal feature of ME/CFS yet current methods of testing, comparing, and monitoring PEM are lacking.

So what few studies WERE included and do they shed light?

There were 36 randomized clinical trial studies for ME/CFS treatment; 9 for medications, 14 for counseling or behavioral therapies, 7 for complementary and alternative medicine, 6 for exercise, and 5 comparing therapies:

• Seven were rated as “good-quality” trials: 5 counseling and behavioral therapies, 2 complementary and alternative medicine therapies
• 24 were rated “fair-quality”
• 5 were “poor-quality”
• The phase III Ampligen trial was rated “fair to good-quality” and it was the only medication trial to get this rating; all other medication trials were fair and poor-quality.
• The PACE trial was rated as “good-quality” for both counseling and behavioral and for exercise intervention but the strength of the evidence was moderate to low for cognitive behavioral therapy (CBT) and graded exercise therapy (GET).
• The other 5 studies that used an exercise intervention were “fair-quality” and overall, exercise intervention trials showed moderate strength effect in function and well being and low-strength effect on fatigue. Importantly, the high rate of refusal in one study may indicate that patients are concerned the possible adverse effects of repeat exercise testing (e.g., the first exercise test may have caused PEM prohibiting the ability to do the second test). This deserves further study and consideration as a possible outcome.

All of the clinical trials were compromised because of small and heterogeneous patient populations, differences in inclusion criteria – including use of different case definitions, and measuring different outcomes (e.g., fatigue, function, etc). This is exactly what Dr. Peter Rowe reported at the 2013 FDA workshop on Development of Safe and Effective Drug Therapies for CFS and ME (http://www.tvworldwide.com/events/fda/130425/). Measuring the effect of an intervention was likely diminished by the above factors. Ways to better detect the effect of an intervention are to measure certain clinical features present before treatment starts compared to after treatment, such as severity and frequency of specific symptoms like brain fog or PEM. This is called subgroup analysis; not one of the intervention studies did a subgroup analysis.

Is there a Silver Lining?

Despite the lack of an evidence-base, this systematic review shines a light on important areas that must be pursued to build the ME/CFS comparative effectiveness research evidence base:

1. Determine diagnostic criteria that can be used as the gold standard for ME/CFS diagnosis.

• We note that the work of determining ME/CFS diagnostic criteria is currently underway at the Institute of Medicine (IOM)

2. Test gold standard diagnostic criteria in other populations with diseases similar to ME/CFS where diagnostic uncertainty exists so that the treatment is specific to ME/CFS vs another similar condition.

• Determine the outcomes that are clinically meaningful and ensure standardized assessment so that the effect of interventions can be measured
• Post-exertional malaise is a cardinal feature of ME/CFS and potentially one of the most important outcome measures; research should be conducted to determine what it is and how to measure it

3. The biomarker research and clinical trials conducted to date provide clues to the possible causes of ME/CFS and can serve as “disease models” for further study of ME/CFS pathophysiology

• All basic and intervention research should use methodological standards to  ensure that it meaningfully contributes to the ME/CFS evidence-base

This analysis illustrates the lack of coherence in the field and the absence of high quality clinical trial data.  NIH, through the P2P workshop, set out to identify gaps in the research, and they found more gaps than substance. Yet the existence of this effort shows increased recognition that ME/CFS is an urgent area of future study and clearly implies that more resources need to be focused on well designed, adequately powered studies. We believe that it is NIH’s responsibility to address their own finding.

What Happens Now?

The P2P panel will review this evidence-based report, including the comments and feedback now being submitted, in advance of the meeting that will take place on December 9 & 10, 2014. At that two-day workshop in Washington DC, the P2P panel will hear from the expert speakers and be able to ask clarifying questions in a town-hall-like Q&A that will take place after each session.

Experts in ME/CFS are being invited to address each agenda item. They will speak to their personal experience and expertise as a patient, caregiver, researcher, etc. The slate of speakers has not yet been released.

The day after the P2P meeting, the P2P Panel will write a draft report which will be published and the public will have time for comment on the final report, just as we are now commenting on the draft evidence review. The comment period is 30 days; the deadline should be around January 12th, 2015. Once the comment period closes, the report will be finalized and NIH will organize a plan to disseminate it widely.

The goal, as we understand it, is a report that contains a set of recommendations based on the totality of the evidence, in the hopes of having said recommendations carried out by Federal partners in 2015. These recommendations are aimed at improving the nature of the research being conducted in ME/CFS. The Solve ME/CFS Initiative will pass along additional information as it is received and when the opportunity for public comment is scheduled, we will report that and share the means by which you can participate.

For far too long, ME/CFS has been underfunded, misunderstood, and even disbelieved. Our community has been pushing a very heavy fly-wheel aimed at increasing understanding, acceptance and progress toward treatments and cure. Now there is federal activity, all aimed at moving the needle for ME/CFS – FDA Voice of the Patient, IOM activity on creating clean diagnostic tools and other much-needed tasks, and this P2P. What this evidence based report shows is that the status quo must change. We hope P2P will illuminate the clear need for more funding, increased research activity and faster progress.

What Can You Do?

Registration is now open for the Pathways to Prevention workshop for Advancing the Research on ME/CFS taking place on December 9 & 10. You can register to attend live or participate via webcast. It is our hope that many stakeholders will participate in this process in order to ensure the patients have a strong presence and a voice.