The Dangerous Language Of Medical Research

Today's post from updates.pain-topics.org (see link below) is another excellent article from Dr Leavitt, talking about how drugs are often regulated based on several vague words of conclusion from research papers. 'There is no evidence to suggest...' can let governments and regulatory bodies completely off the hook in their decision making about drugs availability. It means that patients need to be constantly alert and conscious of the fact that their necessary medication may be refused on very shaky grounds. It may require some foot-stamping to get your way; especially if you have done all the research and have taken expert advice which suggests that a particular drug could help you considerably. It's rejection may hang on those six words but in fact all they mean is the opposite of what they say - there may be no evidence to suggest but that equally means there may be no evidence to prohibit. Making an ally of your doctor and taking well-researched and factual arguments to him or her may convince the powers that be that you will in fact, benefit from taking such a drug. 'There is no evidence to suggest...can often be a cop-out which will worsen your condition. This article is well-worth reading.

The 6 Worst Words in Evidence-Based Medicine
Posted by SB. Leavitt, MA, PhD Saturday, December 14, 2013

Writing in the November 2013 edition of the Journal of the American Medical Association, R. Scott Braithwaite, MD, MS, from New York University School of Medicine, comments on a deceptive 6-word phrase often used in evidence-based medicine (EBM) that frequently leads to dangerously false inferences for clinical decision making [Braithwaite 2013]. We further contend that, applied to the interpretation and application of pain research — such as relating to the use of opioids analgesics for chronic pain — those 6 words also can encourage poor quality pain management and inexcusable patient suffering.

What are the offending 6 words? They are quite simply, “There is no evidence to suggest….” Braithwaite proposes that this phrase should be banished from the lexicon of EBM and, while this could be important, we also recognize that it could foster uncertainty and doubt that could be discomforting to many professionals and patients in the pain field.

Untested Hypotheses

Braithwaite provides the following statements as examples of nefarious usage of the phrase:
“There is no evidence to suggest that hospitalizing compared with not hospitalizing patients with acute shortness of breath reduces mortality.”

“There is no evidence to suggest that ambulances compared to taxis to transport people with acute GI bleeds reduces prehospital deaths.”

“There is no evidence to suggest that looking both ways before crossing a street compared to not looking both ways reduces pedestrian fatalities.”

As Braithwaite maintains, all of the statements are absurd as a basis for decision making, yet each statement is technically correct since its underlying hypothesis has not been suitably tested to establish contradictory evidence. This presumes a definition of “evidence” that requires formal hypothesis testing in an adequately powered (eg, large sized) and well-designed (eg, randomized, controlled) research study.

Taking this further, based on a prior review article [Smith and Pell 2003], we would add the following statement: There is no evidence to suggest that jumping from an airplane in flight without a parachute as compared with using a parachute is fatal. As the review authors note (somewhat satirically), while there have been anecdotal accounts of persons without parachutes surviving falls from airplanes, it is extremely difficult to recruit subjects for good-quality randomized controlled trials comparing parachute use with no parachute in such circumstances; so, the statement is technically correct, but unproven and misleading.

Based on his observations, Braithwaite proposes that “there is no evidence to suggest” has become a mantra for EBM practitioners in a wide variety of settings. And, he says, rarely is the statement followed by the clarifying aphorism “absence of evidence is not evidence of absence” [also see Altman and Bland 1995] or discussions of more inclusive definitions of “evidence” for affirming the hypotheses in question.

Seeking Clarity & Precision

Braithwaite proposes that, when an intervention potentially may incur significant harm or require large commitments of resources, deciding not to intervene when “there is no evidence to suggest” the favorability of the intervention can be prudent. “However, deciding to intervene when ‘there is no evidence to suggest’ also may make sense,” he writes, “particularly if the intervention does not involve harm or large resource commitments, and especially if benefit is suggested by subjective experience (eg, the qualitative analogue of the Bayesian prior probability).” 

Side Note
Bayesian theory applied to medical research is regaining popularity — albeit, it is difficult for most people to understand — and it provides a mathematical framework for inference or reasoning using probability estimates. The approach can be particularly helpful in judging the relative validity of hypotheses in the face of sparse or uncertain data. While actual calculations can be complex, to evaluate the probability of a hypothesis being “true” an investigator specifies a prior probability — based on current observation/experience, past research, or scientific principles — which is then updated in the light of new, relevant research data to provide a posterior probability (ie, outcome result). An important feature of a Bayesian approach is that it takes into account what already is known or can be estimated, either quantitatively or qualitatively, about the likelihood of research outcomes being valid; if there is absolutely no (zero) prior probability to support a hypothesis, then research outcomes — whether favorable or unfavorable — are usually unlikely to be valid and reliable. In many respects, this might be viewed as a statistical application of the “Bradford Hill Criteria” for establishing cause-effect relationships [as discussed in Part 11 (here) of our series on “Making Sense of Pain Research].

Braithwaite further maintains that a fundamental problem with the phrase “there is no evidence to suggest” is that it is “ambiguous while seeming precise.” What does “there is no evidence to suggest” really mean when used to argue against some intervention?

Does it mean that the intervention has been proven to have no benefit? That some evidence does exist, but it is inconclusive or insufficient? That outcomes are somewhat equivocal, with risks exceeding benefits for some patients but not others? Each has a subtly different meaning affecting decision making; whereas, simply stating “there is no evidence to suggest” circumvents the experience or clinical intuition of healthcare providers. Furthermore, as Braithwaite notes, many decisions are particularly sensitive to patient preferences and, when the favorability of an intervention is unclear, “there is no evidence to suggest” may “inhibit shared decision making and may even be corrosive to patient-centered care.”

According to Braithwaite, most practitioners make patient-centered decisions every day without high-quality (eg, randomized controlled trial) evidence as a guide, and those decisions are not always wrong. Furthermore, principles of EBM make it clear that an evidence-based approach was never intended to entirely exclude information derived from clinical experience and intuition — which amounts to a qualitative prior probability in a Bayesian sense.

He recommends that practitioners and researchers make concerted efforts to banish “there is no evidence to suggest” from their professional vocabularies. Instead, they could substitute one of the following 4 phrases, each of which has clearer implications for decision making:
“Scientific evidence is inconclusive, and we don’t know what is best” (corresponding to an uninformative or ambiguous Bayesian prior probability).

“Scientific evidence is inconclusive, but my experience or other knowledge suggests ‘X’” (corresponding to an informative, qualitative Bayesian prior probability supporting ‘X’).

“This has been proven to have no benefit” (if valid evidence indeed exists to confirm this).

“This is a close call, with risks exceeding benefits for some patients but not for others.”

Braithwaite asserts that each of the 4 statements would lead to distinct inferences for decision making and could improve clarity of communication with patients. Finally, he says, “Informed implementation of EBM requires clearly communicating the status of available evidence, rather than ducking behind the shield of 6 dangerous words.”

False Arguments Over Opioids for Chronic Pain

For quite some time, a very outspoken and opinionated group of healthcare professionals in the United States has been arguing against the long-term use of opioids for chronic noncancer pain, based essentially on the premise “there is no evidence to suggest that the benefits of this therapy outweigh its potential risks.” In fact, going further — by relying on similar logic and bolstered by low-quality, invalid, or misinterpreted evidence — they assert that overwhelming risks negate any benefits. The group also went so far as to petition the FDA to make the labeling of all extended-release (ER) and long-acting (LA) prescription opioids more restrictive [first discussed in an UPDATE here]. Even though the petition’s demands were largely rejected by the FDA in updated product-labeling [see UPDATE here], opioid opponents have persisted in their campaign.

Indeed, it is acknowledged that there is virtually no clinical research evidence of good quality examining the efficacy and safety of opioid analgesics prescribed long-term for chronic pain. And, in their labeling-change mandates, the FDA also requires manufacturers to conduct longer duration trials of ER/LA-opioids, including evaluations of serious risks, such as misuse, abuse, addiction, overdose, and death, as well as the risks of developing increasing sensitivity to pain (hyperalgesia).

Meanwhile, the opioid opponents have been using the current lack of evidence as evidence itself to support what might be called argumenta ad ignoratum, or “appeals to ignorance,” as discussed in Part 12 of our “Making Sense of Pain Research” series [here]. In the absence of any high-quality research evidence to the contrary, the opponents have used their own interpretations of data on opioid-related abuse, addiction, deaths, and other risks to arrive at an artificial Bayesian prior probability of harm — and have successfully foisted fallacious inferences on the public.

Additionally, they are most likely driven by a personal set of prior probabilities — coming from likeminded peers or individual experiences with select patients — that help guide the calculus of their conclusions. Essentially, they have fabricated their own rendition of Braithwaite’s second statement above to claim, “Scientific evidence is inconclusive, but my experience or other knowledge suggests that opioids are ineffective and unsafe in the treatment of chronic noncancer pain.”

However, using similar evidence deficits and prior probabilities informed merely by empiricism (eg, anecdotal observations), there are other important arguments about opioids for chronic pain that can be stated:
There is no evidence to suggest that opioid-induced hyperalgesia is a frequent clinical occurrence in human subjects administered opioids long-term for any type of pain, or which patients might be most affected.

There is no evidence to suggest that there is an inordinately high incidence rate of de novo, iatrogenic addiction among patients with chronic pain prescribed long-term opioid analgesics.

There is no evidence to suggest that significant numbers of patients with chronic pain do not or cannot benefit from opioid analgesia.

Other, similar, arguments could be expressed that cast doubts on concerns about the efficacy and safety of opioids for chronic pain. But, in all cases, such doubts are motivated by uncertainty — or, an “ambiguous Bayesian prior probability” — and a most objective and unbiased premise could be a variation of Braithwaite’s first statement above; “Until there is good-quality evidence available we cannot reach definitive conclusions.” Meanwhile, using a lack of evidence to argue for or against opioids for chronic pain becomes a cruel game of sorts in which nothing is scientifically established and patients who presently do or prospectively could benefit from such therapy are the losers.

Ubi Dubium, Ibi Intellectum

If we accept Braithwaite’s proposal to eschew the use of “there is no evidence to suggest” as a valid argument against a therapy or intervention, it also raises nagging doubts about the legitimacy of rejecting certain questionable modalities for pain management because they have little if any high-quality evidentiary support. A number of complementary and alternative medicine (CAM) modalities immediately come to mind: eg, homeopathy, reflexology, energy-field therapies (eg, Reiki, etc.), biomagnetic therapy, some variations of acupuncture, and others.

In most cases, high-quality clinical trials are absent and we are left with observational or anecdotal evidence at best. With certain approaches (eg, homeopathy, Reiki, and others), there is no presently-known biological rationale or plausibility to serve as a prior probability of efficacy. Still, there are ample examples of patients with pain being helped by each of the treatments — an informative prior probability — even if the outcomes are primarily due to placebo effects. So, should those CAM approaches be rejected outright as worthless on the basis of “there is no evidence to suggest that they are clinically effective for pain”?

Indeed, many critics have made strong, rational arguments for unequivocally rejecting most CAM approaches on the basis of absent or inadequate supportive evidence and/or the lack of biological plausibility [eg, see Science Based Medicine blog]. Despite those contentions, and in view of Braithwaite’s perspective, it would appear that less absolutist and more definitive statements are needed. And, these must not rely primarily on the absence of evidence as evidence against CAM approaches and any prior probabilities must be taken into account, including those based on limited observational or anecdotal data.

In many cases, prior probability or plausibility may be so low that the respective CAM approach is still deemed ineffective. But, in other instances, this could encourage “suspended disbelief” until further investigation via high-quality research is possible. Many practitioners and patients may be discomforted or irritated by the degree of uncertainty and doubt this tolerates. And, a dilemma may be that, as with the parachute example above, there may never be definitive research to make strictly evidence-based pain management decisions. However, a fundamental theme of these UPDATES, as well as our “Making Sense of Pain Research” educational series has been Ubi Dubium, Ibi Intellectum, or “Where There Is Doubt, There Can Be Understanding” [click to download series PDF].

REFERENCES:
> Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ. 1995;311(7003):485 [access here].
> Braithwaite RS. EBM’s Six Dangerous Words. JAMA. 2013;310(20):2149-2150 [access by subscription here].
> Smith GC, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ. 2003;327(7429):1459-1461 [abstract here]

http://updates.pain-topics.org/2013/12/the-6-worst-words-in-evidence-based.html