Death By Anecdote, Part 2

Please read the first part of this article.

When a story is false, we run into a third problem: Even when misinformation is corrected, many people continue to believe it. For example, in one study, participants from Australia, Germany and the United States were asked about arguments made by the US government favoring the invasion of Iraq which were subsequently retracted, such as the claim that Iraq possessed weapons of mass destruction (WMDs). The Australian and German students were sensitive to retraction; that is, when they knew a claim had been retracted, they tended not to believe it. The Americans, however, were insensitive to retraction; on the whole, they tended to believe statements that they knew had been retracted about as much as statements they did not know had been retracted.

Paradoxically, attempts to correct misinformation can lead to a backfire effect, in which people are more likely to believe false information after it has been debunked. Sometimes this is due to increased familiarity with the claim as a result of its being repeated during the retraction. However, backfire effects are most likely to occur when the original claim is consistent with the ideology of the person who believes it. Not surprisingly, Republicans were more likely than Democrats to continue to believe that Iraq had WMDs even after the Bush administration admitted they didn't. When ideological backfire effects occur, people can be suspicious of the motives of the person or organization doing the correcting (often the news media) and discount the retraction.

Lewandowsky and others make three suggestions for successfully correcting misinformation: (1) warn people at the time of initial exposure that the information is suspect, (2) repeat the retraction several times, focusing only on the new, correct information, and (3) provide a plausible alternative explanation for the previous false belief. In the case of ideologically motivated false beliefs, they make a fourth recommendation: Affirm the target's ideology before attempting to correct the false belief. The first two remedies require news media cooperation that the Obama administration is unlikely to receive. Since no single alternative explanation accounts for all the anti-ACA anecdotes, providing plausible explanations for false beliefs requires extensive investigation of individual cases. The fourth suggestion would seem to require a statement like this: “We agree with you that Obamacare is a disaster, but in this case, you are wrong because . . .” 

Here is Lewandowsky discussing misinformation and its correction in the important area of climate change.

I see no magic bullet here. The best solution for the administration may be to appeal to the value Americans place on self-reliance and encourage them to explore their health care options for themselves. But they can't do that until their website is fixed. If they lose the cooperation of young people, the market will suffer from adverse selection—not enough people paying into the system, and too many older, sicker people drawing it down. Insurance rates could increase dramatically next year, just in time for the 2014 elections. Then Obama will see how much fun it would be if the Tea Party controlled the Senate as well as the House.

The HealthCare.gov fiasco is bad news for single payer advocates too. I'm afraid most people don't realize that it was Obama's reliance on an "overly-complicated, market-based Republican health care plan" that made the website so difficult to set up. They may simply conclude that government can't do anything right.

You may also be interested in reading:

Death By Anecdote, Part 1

Policy By Anecdote

Climate of Mistrust

The Open and Closed Mind

More than three years after Sarah Palin made the false claim that the Affordable Care Act included “death panels” that would decide whether people would receive life-saving medical treatment, 40% of Americans still believe it. A new experiment by Brendan Nyhan, Jason Reifman and Peter Ubel considers the death panel myth in the context of the perseverance effect, a cognitive error in which people's beliefs about reality persist even though the evidence on which they were based has been discredited.

In one perseverance study, participants tried to distinguish between real and fake suicide notes. In fact, regardless of what they said, half the people were told they were right most of the time (the “success” condition), while the others were usually told they were wrong (the “failure” condition). Later, the participants were debriefed. It was explained that the feedback was false and that they had been randomly assigned to success and failure conditions. Even though they understood the debriefing, the “successful” people still believed they were better at the task than those who had “failed” thought they were.

Perseverance affects people's reactions to scientific studies that support or contradict their beliefs. Another experiment involved students with contrasting attitudes, some who believed that capital punishment was a deterrent to murder and others who rejected this claim. People in both groups read summaries of two fictitious studies, one which supported the deterrence hypothesis and one which didn't. You might think reading mixed evidence would lead people to moderate their beliefs. However, both pro- and anti-deterrence participants became more convinced of their original beliefs—a backfire effect. The two groups' beliefs became more polarized.

Perseverance studies seem to suggest that fact-checking politically biased claims is a losing effort. Recent research suggests that more educated and well-informed people show greater perseverance, since having more information allows them to explain away contradictory findings more easily. The Nyhan study is in that tradition. Here is Dr. Ubel explaining the study.

The chart below shows the results. On the left are the low knowledge participants. Without the correction, those who liked Palin believed the death panel myth, but the rebuttal was effective in disabusing them of this belief. However, the high knowledge Palin supporters believed the death panel myth more with the correction than without it. The correction backfired.

Just a word of caution. I was fully prepared to believe this study; however, the more I thought about it, the more I sympathized with the high information Palin supporters. The rebuttal to Palin's claim read (in its entirely) as follows:

However, non-partisan health care experts have concluded that Palin is wrong. The bill in the House of Representatives would require Medicare to pay for voluntary end-of-life counseling sessions, but there is no panel in any of the health care bills in Congress that judges a person's “level of productivity in society” to determine whether they are “worthy” of health care.

Should a well-informed Republican really be satisfied with this rebuttal? If I were in their shoes, I might want to ask some additional questions. Who are these “non-partisan experts?” The correction addresses Palin's claim about end-of-life counseling, but what about the Patient-Centered Outcomes Research Institute, which is charged with evaluating the effectiveness of medical treatments, and the Medicare Independent Payment Advisory Board, which is supposed to refuse to pay for expensive but ineffective treatments? How much indirect control will they have over the treatment patients actually receive? I personally hope these groups will help to eliminate wasteful procedures, but since they haven't met yet, we don't really know.

I'm not questioning the existence of perseverance, or that it makes researchers' lives a lot more difficult. However, another way to read this study is that well-informed people may require more detailed counterarguments than these authors provided.

Motivated Reasoning

The human understanding when it has once adopted an opinion draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises, or else by some distinction sets aside and rejects, in order than by this great and pernicious predetermination the authority of its former conclusion may remain inviolate.

Francis Bacon

Monday's Pittsburgh Post-Gazette contains two articles about drilling for natural gas in the Marcellus shale by the method known as fracking. One of them reports that the U. S. Chamber of Commerce is beginning a new lobbying effort in order to persuade the citizens of Pennsylvania, Ohio and West Virginia not to annoy the natural gas industry by enacting new taxes or environmental regulations. The Chamber will spend “millions of dollars” on this campaign. (The actual amount is not stated.) Gene Barr, president of Pennsylvania's Chamber of Business and Industry, said, “You can call it advocacy or lobbying, but I use the word education.” Presumably, the campaign will be at least as educational as those slick Range Resources commercials that tell us how much a few of our fellow citizens have profited from natural gas royalties.

It would seem that this campaign is a waste of money here in Pittsburgh, since the state of Pennsylvania is already a wholly-owned subsidiary of the natural gas industry, and has given it the most industry-friendly legislation in the country. I guess if you're drilling for natural gas, you can never be too careful.

Elsewhere in the same newspaper, under “National Briefs,” I found an op-ed masquerading as a news article that might easily have been part of the Chamber of Commerce's new campaign. Under the headline, “Experts dispute fracking critics,” it carefully selects paragraphs from the beginning and near the end of this Associated Press article. The original article is already very pro-industry, using selective citation of research to brand several claims made by opponents of fracking as false. The edited version removes what little balance the original contained by removing the last two paragraphs, which report false statements by the natural gas industry.

What caught my eye was a reference to social psychology in order to justify the claim that fracking opponents have fallen victim to self-delusion. Political scientist Mark Lubell attributes the problem an “an actual psychological process” called motivated reasoning, in which people “insist on believing things that aren't true, in part because of feedback from other people who share their views.” 

As a psychological term, “motivated reasoning” is frustratingly vague. It tells us almost nothing about process. But it is true that, since Leon Festinger published A Theory of Cognitive Dissonance in 1957, social psychologists have conducted thousands of studies showing that people are more likely to accept information that supports their prior beliefs than information that contradicts them. Clearly this is something that we all do a lot of the time. Here's journalist Chris Mooney explaining motivated reasoning and explaining one of the processes involved.

Brendan Nyhan and Jason Reifler did four studies designed to test whether media reports can be expected to correct false beliefs. Participants read two mock news articles—a false claim, followed by an article which corrected it. For example, one study presented President George W. Bush's false claim from after the invasion of Iraq that Saddam Hussein had possessed weapons of mass destruction, followed by an article about the Duelfer Report that refuted that claim. All four studies found evidence of motivated reasoning, in that the corrections did not significantly reduce the original misperceptions in their overall sample.

More importantly, two of the studies found evidence of a “backfire effect,” in which the correction increased belief in the original false claim, among certain subgroups. Subjects were asked to classify themselves on a 7-point scale of liberalism-conservatism. In the WMD study described above, the liberals and centrists either corrected their false beliefs or were unchanged, but the conservatives became more extreme in their belief that WMDs had been found. However, this backfire effect failed to occur when the study was repeated.

The backfire effect was replicated, however, in a second study in which participants were first told that tax cuts would stimulate the economy and increase government revenue, and then told that research had shown this claim to be false. Again, it was the conservatives who increased their belief in the efficacy of tax cuts after being shown studies that found them to be ineffective. The fourth study was conducted in order to try to find a backfire effect among liberals. Subjects were falsely told that President Bush had completely banned stem cell research, and later correctly told that he had only banned federally-funding research, but allowed privately-funded research to continue. In this case, participants once again persevered in their original beliefs, but no backfire effect was found.

Here's Brendan Nyhan discussing this research and making some suggestions as to how journalists can correct their audience's false beliefs.

The Nyhan-Reifler studies seem to suggest that conservatives are more likely to engage in motivated reasoning than liberals. In fact, after reviewing the entire literature on this subject, Chris Mooney comes to this conclusion. However, we need to be extremely skeptical before accepting this hypothesis. It's certainly possible that liberals would show a backfire effect if the study involved an issue that was more central to their political ideology.

Here's the problem with the fracking article. It's far too easy to read the motivated reasoning literature selectively and use the concept as a weapon to attack your political opponents, thereby engaging in motivated reasoning about motivated reasoning. The claim of motivated reasoning should only be used when the argument someone is defending has been clearly refuted by the evidence. The AP article fails to distinguish between claims that are false and those that are merely unsubstantiated. A claim can be unsubstantiated if it has not been conclusively tested, or if the available evidence is mixed.

While it is certainly true that some opponents of fracking have made false or irresponsible accusations, the claims that fracking will contaminate the air or ground water, or that it will expose people to cancer-causing chemicals, are plausible and serious enough to justify independent—that is, not industry-funded—research at multiple locations. The argument that natural gas is a cleaner burning fuel than coal has been questioned by highly competent research. This debate is further complicated by the fact that it may take decades for conclusive evidence of some harms caused by fracking to emerge.

Meanwhile, we should not be surprised if our corporate media, which are heavily dependent on advertising by the fossil fuel companies, treat industry propanganda as if it were unbiased and accuse those who question it of motivated reasoning.