The Matthew Effect

The Matthew effect is a term introduced by sociologist Robert Merton in 1968. It takes its name from Matthew 25:29, the parable of the talents:

For unto everyone that hath shall be given, and he shall have abundance; but from him that hath not shall be taken even that which he hath.

In other words, the rich get richer and the poor get poorer. Merton suggested that the positive behaviors of high status people are more likely to be recognized and rewarded than those of low status individuals, while high status people's mistakes are more likely to be overlooked. This creates a positive feedback loop in which increased confidence causes their performance to improve and their reputation to increase over time. The opposite happens to low status individuals. Their mistakes are more apparent, leading to negative feedback, stress and disruption of performance.

Merton also coined the term self-fulfilling prophecy, in which predictions result in behaviors that cause the predicted outcome to occur.  The Matthew effect is a type of self-fulfilling prophecy in which the observer's positive (or negative) expectations cause more (or less) successful behavior in the target over time. This has broad implications for people's self-esteem and the inequality of their social and economic outcomes.

Two business school professors, Jerry Kim and Brayden King, looked for evidence of the Matthew effect in major league baseball. They predicted that a pitcher's status would influence calls by the home plate umpire. Pitcher status was defined as the number of times he had previously been chosen to the All-Star team. It was predicted that, as the number of All-Star appearances by a pitcher increased, more of their balls would be called strikes (over-recognition) and fewer of their strikes would be called balls (under-recognition). The study was made possible by the Pitch f/x system, in place in all major league ballparks, in which cameras objectively measure whether each pitch is in the strike zone.

© www.sportvision.com

The data base was all the pitches taken (not swung at) by the batter during every game of the 2008 and 2009 seasons. These pitches must then be called either a ball or a strike by the umpire, and each call was evaluated for correctness. These data were related to over two dozen pitcher, batter, catcher, umpire and situational characteristics. Some of these variables are of real importance to baseball fans, but they could all be statistically controlled in order to evaluate the status hypothesis.

Baseball fans may be interested in the big picture—the distribution of correct and incorrect calls among the almost 800,000 calls the researchers measured.

	Called Ball	Called Strike
Actual Ball	87.10%	12.90%
Actual Strike	18.80%	81.20%

The umpires were correct about 85% of the time. (There were more actual balls than actual strikes.) Umpire bias favored the batter, since more strikes were called balls than balls were called strikes. The count (the number of balls and strikes to that point) had a big effect. For example, the likelihood that the umpire mistakenly called a strike was 62% lower when the count was 0-2 and 49% higher when the count was 3-0. Apparently, umpires don't like their call to end an at-bat. Umpire calls also tended to favor the home team. Errors of both over- and under-recognition increased with the situational importance of the at-bat.

The hypothesis was strongly confirmed. Look first at over-recognition: Holding all other variables constant, the more trips a pitcher had made to the All-Star game, the more likely a ball was to be called a strike. The probability of a mistaken strike call increased from 12.8% among pitchers who had no All-Star appearances to 14.9% among pitchers with five or more appearances. Each additional trip to the All-Star game increased the likelihood of over-recognition by 4.9%.

The situation was reversed for under-recognition, also confirming the hypothesis. A strike thrown by a pitcher with no All-Star appearances was mistakenly called a ball 18.9% of the time, but only 17.2% of the time if the pitcher had five or more appearances. Each trip to the All-Star game decreased the likelihood of under-recognition by 2.7%.

In further analyses, the authors were able to show that, with this large data set, pitcher status also had statistically significant effects on the outcome of the at-bat (the total bases reached by the batter) and the game (whether the pitcher's team won). In an analysis that made some admittedly questionable assumptions, they calculated that umpire errors alone were worth approximately $575,000 in salary to a high status pitcher over the course of his career.

© totallycoolpix.com

Of course, Matthew effects can occur any time one person evaluates another—a teacher grading a student, a boss rating a worker, a reviewer reading a manuscript, etc. As a demonstration of how quickly performance expectations can occur, consider a study by Ned Jones and others. Participants watched a videotape of a college student answering 30 difficult questions, with feedback after each item indicating he had answered 15 of them correctly. In the ascending condition, the student gradually improved. He got three of the first ten right, five of the second ten, and seven of the last ten. In the descending condition, the pattern was reversed. (The difficulty of the questions was held constant by asking exactly the same questions in the opposite order.) First impressions mattered a great deal. The student was rated as more intelligent in the descending than in the ascending condition. The authors had hoped the ascending student would get some credit for improvement, but it didn't happen.

In this experiment, as in baseball, the expectations were based on the target's actual past performance. However, expectations can be based on gender, race, class or other social categories. In other words, stereotypes based on group membership can create self-fulfilling prophecies leading to discrimination.

There is no reason to think that umpires and ballplayers are consciously aware of the systematic nature of these errors. A New York Times article about the Kim and King study included the usual quotes from baseball people expressing their surprise at or disbelief in the results. Most teachers, bosses and reviewers probably think they're being objective, too.

In major league baseball, the technology is already in place to have balls and strikes called automatically using the Pitch f/x system. Why would anyone (except maybe Clayton Kershaw) not think that's a good idea?

You may also be interested in reading:

Is Democracy Possible? Part 1 (see also Parts 2 and 3)

Deja Vu

Does Money Make You Mean?

Headline News

Although newspapers get most of their national and international news from larger newspapers or wire services, headlines are written locally either by editors or headline-writing specialists. Friday's (June 6) front page of the Pittsburgh Post-Gazette contains two headlines I consider misleading.

The lead story was about General Motors' internal investigation of why it failed to recall cars with faulty ignition switches for over a decade, which caused several deaths. The investigation led to the firing of 15 GM employees. The headline reads, “GM Places Blame in Recall Mix-up.” One would expect GM's explanation of its own behavior to be self-serving, but the headline, by referring to the scandal as a “mix-up,” gives it an even more benign interpretation than GM's own corporate spin doctors, who, according to the article, cited “a pattern of 'incompetence and neglect.'”

A second article concerns long waiting times for medical care at the Veterans Administration. The headline states, “18 More Deaths Tied to VA Wait List.” The problem with this headline is the words “tied to.” As the article explains, it is not yet known whether longer waiting times contributed to these newly discovered deaths. This headline commits the opposite error of the GM headline by making the scandal appear worse than it is presently known to be.

These two cases are consistent with the possibility that, when wrongdoing is suspected, newspapers give corporations the benefit of the doubt, but are willing to believe the worst about government agencies. Of course, the VA doesn't advertise in the Post-Gazette the way Chevy dealers do.

The worst headlines are those that imply the exact opposite of the article. On May 16, we read, “FCC Pushes Net Neutrality Proposal.” This is technically correct, since the FCC's proposal deals with the issue of net neutrality, but I suspect most readers would infer from this headline that the FCC was proposing a policy of net neutrality. Only by reading the article do we find that they are actually proposing an end to net neutrality by allowing internet service providers to charge companies such as Netflix higher prices for faster data transmission, leaving the rest of us with slower service and potentially higher prices as these companies pass on their costs to the consumer.

© guide.cred.columbia.edu

Headlines are important.  Not only do people use them to decide whether to read the article, but they create a first impression that can influence the reader's interpretation of the article. Psychologists call this the prior entry effect. Early information about an object or event has a greater effect on our final impression than later information. Media theorists call it a framing effect. A headline frames a news story by creating an expectation about what information is to follow. Expectations can easily become self-fulfilling prophecies when we see what we expected to see. The word "mix-up" implies that the actions of GM employees were unintentional, even though no event described in the article conforms to the everyday meaning of a "mix-up," in which one object or piece of information is mistaken for another.

How do misleading headlines get written? My guess is that, under time pressure, newspaper employees don't always read the articles carefully, but write headlines based on their own expectations of what will be in them. These examples imply that more time and thought should be given to headline-writing.

You may also be interested in reading:

In Denial

Motivated Reasoning

The Media Select Their Candidate

In Denial

Political scientist Brendan Nyhan was on All In with Chris Hayes last night (March 11) for a discussion of the Julia Boonstra television ad. The political ad, which was financed by the Koch Brothers' PAC, Americans for Prosperity, is running in Michigan in an early attempt to discredit Democratic Senatorial candidate Gary Peters. In it, Ms. Boonstra, a cancer patient, claims that her health insurance was cancelled due to the Affordable Care Act, that she can't afford the alternative policy offered to her, and that she is afraid she will die as a result. The ad was awarded three Pinocchios by the Washington Post after it was discovered that the alternative policy actually would save her $1200 a year. Her response was, “I personally do not believe that,” and that it “can't be true.”

This ad is part of a larger campaign in which Republican organizations trot out Obamacare “victims” to tell stories that are almost always misleading. Journalists eventually debunk these stories, but many more people hear the lie on television than read the truth in a newspaper, and even those who are told that the story is false often refuse to believe it.

Brendan Nyhan has been doing research for several years which shows that attempts to correct misinformation fail when they oppose people's deeply entrenched political ideologies. In fact, they can backfire and produce a boomerang effect—attitude change in the direction opposite to that intended by the speaker. Dr. Nyhan's latest study, released just last week, was an attempt to dispel the myth popular among conservatives that the measles, mumps and rubella vaccination (MMR) causes autism. Here's some background on the controversy from Aaron Carroll.

In the study, a nationally representative sample of 1759 young parents were randomly assigned to receive one of four pro-vaccine messages or a control message, delivered via the internet. The pro-vaccine messages, all taken from the Center for Disease Control (CDC) website, were:

1. Autism correction—written scientific evidence debunking the vaccination-autism link.
2. Disease risks—written descriptions of symptoms and risks associated with measles, mumps and rubella.
3. Disease narrative—a dramatic narrative in which a mother tells how her baby almost died from the measles.
4. Disease images—photographs of children suffering from symptoms of the three diseases.

The control group received an article about the costs and benefits of bird feeding. Nyhan measured the belief that the MMR causes autism, the belief that the MMR has serious side effects, and whether the participants intended to have their next child vaccinated, both before exposure to one of the messages and again about two weeks later.

The results were disappointing. Compared to the control group, the autism correction message significantly reduced the belief that the MMR causes autism. That was the only intended effect. The other three significant results were all boomerang effects. The disease narrative increased the belief that the MMR has serious side effects. The disease images increased the belief that the MMR causes autism. And finally, the autism correction resulted in parents reporting that they would be less likely to vaccinate future children. This latter effect was strongest for those parents who initially believed that the vaccine causes autism.

Ordinarily we should not make too much of studies that demonstrate no significant change. After all, the CDC messages could have been ineffective for a variety of reasons not immediately apparent to a reader of the study. It's also possible that different messages or combinations of messages might have worked. However, these were statistically significant backfire effects, and they should probably be taken seriously. The tendency of beliefs to persist or grow stronger even when the evidence for them is shown to be false is known as belief perseverance. The widely-accepted explanation for perseverance effects is self-persuasion. When people hear a message they disagree with, they counterargue with it. In the process, they expose themselves to their own counterarguments and persuade themselves to change their attitudes in the opposite direction from the message.

In the short run, at least, social scientists are becoming known in the media for delivering a depressing message: Factual information is ineffective in changing people's ideologically-motivated attitudes. So far, we have been unable to propose effective strategies for breaking through the barriers imposed by politically-inspired misinformation.

You may also be interested in reading:

Death By Anecdote, Part 1

The Open and Closed Mind

Motivated Reasoning

Sy Hersh's Bombshell

A lot of reporters write articles purporting to expose secrets of the American intelligence community, but this is different. The article is written by Seymour Hersh, the country's foremost investigative journalist. He began his career in 1969 by exposing the My Lai massacre in Vietnam and its coverup. Younger folks will know him for his reporting of American torture at the Abu Ghraib prison complex in Iraq.

In an article appearing today in the London Review of Books, Hersh accuses the Obama administration of “cherry picking” intelligence to support the conclusion that the Syrian government of Bashar al-Assad was responsible for the chemical weapons attack in Eastern Ghouta last August, said to have killed somewhere between 280 and 1400 people. This conclusion was used to justify the threat of US bombing of Syria. His article begins this way:

Barack Obama did not tell the whole story this autumn when he tried to make the case that Bashar al-Assad was responsible for the chemical weapons attack near Damascus on 21 August. In some instances, he omitted important intelligence, and in others he presented assumptions as facts. Most significant, he failed to acknowledge something known to the US intelligence community: that the Syrian army is not the only party in the country's civil war with access to sarin, the nerve agent that a UN study concluded—without assessing responsibility—had been used in the rocket attack.

That other party is al-Nusra, a group affiliated with al-Qaeda, that is part of the coalition of rebel groups trying to overthrow Assad that the US government supports.

Hersh doesn't say Assad was not responsible for the attack, just that the US doesn't have enough evidence to threaten to start a Middle Eastern war. In effect, he is saying the Obama administration was as foolish and/or dishonest as the George W. Bush administration was during the buildup to the American invasion of Iraq. You might say it's deja vu all over again.

You should read the article, but I'll try to summarize: The US had wiretaps intended to monitor coversations of the Syrian high command, and sensors in the ground near chemical weapons facilities designed to detect any movement of these weapons. The wiretaps, and possibly the sensors, were discovered and neutralized sometime earlier in the year, so if the Syrians carried out the attack, the US was taken by surprise. Two days after the attack, the US used computer keywords to analyze several thousand radio communications from the Syrian army, looking for evidence of the attack. In other words, they started with the assumption that the Syrian government was responsible and look for confirming evidence—a clear example of confirmatory bias. They then assembled a scenario of what the Syrian army would have done if they had carried out the attack, based in part on a training exercise that the Syrians carried out the year before. This scenario was presented as if the US had monitored it in real time, which of course they did not. Hersh says the US also ignored evidence that the shells said to have been used to deliver the sarin had a shorter range than they claimed.

At the same time, another suspect, al-Nusra, has emerged. Hersh cites US intelligence sources claiming to be certain that al-Nusra has sarin, and others stating that the evidence is inconclusive. Those who claim al-Nusra does have sarin refer to the presence in their ranks of Ziyaad Tariq Ahmed, a chemical weapons expert from Iraq—a bit of possible  “blowback” from the Iraq War. Needless to say, the US didn't try to construct a case implicating al-Nusra. They probably didn't have the necessary data.

As journalist Marcy Wheeler points out, Hersh's story helps to answer such questions as why our European allies did not support our threat to attack Syria, and why Obama surprisingly backed off from the threat, agreeing to take the question to Congress, where he faced almost certain defeat, and eventually accepting the Russian plan for dismantling Syria's chemical weapons that is now being carried out. As Hersh puts it:

Do we have the whole story of Obama's willingness to walk away from his 'red line' threat to bomb Syria? He claimed to have an iron-clad case but suddenly agreed to take the issue to Congress, and later to accept Assad's offer to relinquish his chemical weapons. It appears possible that at some point he was directly confronted with contradictory information: evidence strong enough to persuade him to cancel his attack plan, and take the criticism sure to come from Republicans.

Hersh's article appears in a journal unlikely to be encountered by many Americans. (Why was it not published by The New Yorker, his usual outlet?) It will be interesting to see how many American media cover this article, and how it is covered. The corporate media usually jump on any story likely to embarrass the President. But in this case, the story conflicts with their usual unquestioning loyalty to the American security state. We'll probably hear a lot of anonymous denials coming from Washington in the next few days.

Update (12/10):

The corporate media have been largely silent about Hersh's article.  The exceptions are interviews he did on Democracy Now (see the interview below, in two parts) and CNN. As expected, a national intelligence spokesperson denied everything.

The most interesting facts to emerge so far are that the piece was turned down by both The New Yorker and The Washington Post.  The Post told Hersh the sourcing of the article did not meet their standards.

You may also be interested in reading:

Deja Vu

A Stroll Down Memory Lane

Is Democracy Possible? Appendix

To make sense of this post, you must first read Part 1 and Part 2.

Kahan and his colleagues did not discuss differences between the responses of liberals and conservatives to his four contingency problems, in spite of the fact that political ideology interacted with his other variables in a way that was statistically significant.

The asymmetry thesis, most strongly identified with journalist Chris Mooney, states that politically motivated reasoning is greater among conservatives than among liberals. Social psychologist John Jost and his colleagues present evidence that conservatism is more strongly correlated with measures of dogmatism and resistance to change than liberalism.

There is evidence of ideological asymmetry in the Kahan experiment. Here it is:

Table 5

These curves represent the distributions of the responses in each condition. The high point of each distribution is the mean. To help us keep these findings straight, Kahan has drawn the liberals in blue and the conservatives in red. The top two figures are of little interest. They merely show that liberals and conservatives are equally likely to answer the questions about the skin cream incorrectly.

However, the bottom two graphs show ideological polarization. Specifically, the conservatives are more likely than the liberals to answer the question correctly when the correct answer corresponds with their expectations, and they are more likely than the liberals to answer incorrectly when the correct answer conflict with their ideology. This is true of both the low numeracy participants (on the left) and the high numeracy participants (on the right). Stated simply, the two red distributions are farther apart than the two blue ones.

Of course, since this study involves only a single issue, another way of interpreting these data is that conservatives care more about gun control than liberals do.

Table 6

You can also see this ideological asymmetry in the bottom figure of Table 3, repeated above, where, at most levels of numeracy, the two red lines are farther apart than the two blue lines. In a blog post, Kahan acknowledges this evidence of asymmetry, but contends that it less interesting than the fact that both liberals and conservatives show confirmatory bias at high levels of numeracy. He points out that the greatest evidence of asymmetry occurs at low to moderate levels of numeracy (inside the solid circle). At the very highest levels of numeracy (inside the broken circle), the two conservative groups appear to be converging, while the two liberal groups are diverging.

He's right about that. Committed liberals must acknowledge the disappointing performance of the highly numerate liberals. The only reasonable explanation is that bright, highly educated liberals are biased too. But that doesn't change the fact that, in this experiment, conservatives show more evidence of ideological bias than liberals do.

You may also be interested in reading:

Is Democracy Possible?  Part 1

Is Democracy Possible?  Part 2

Climate of Mistrust

The Open and Closed Mind

Motivated Reasoning

Is Democracy Possible? Part 2

You must read Part 1 before continuing or none of this will make sense.

A study was conducted by Dan Kahan of Yale University and three colleagues involving a demographically representative sample of 1111 American adults recruited by Polimetrix/YouGov, an online survey research firm. Preliminary questions were used to categorize the participants along two dimensions:

• Liberalism v. Conservatism: They rated themselves as very liberal to very conservative on a 5-point scale, and strong Democrat to strong Republican on a 7-point scale. These scales were combined, and the participants were divided at the midpoint into liberal Democrats and conservative Republicans.
• Numeracy: Participants' numeracy was measured using nine real world mathematical problems. They received a numeracy score of 0 to 9.

Each participant was then asked to answer one of the four judgment of contingency problems shown in Table 2 of Part 1. These are labeled by the correct answer to the problem:

• Rash increases: The rash got worse when the skin cream was used.
• Rash decreases: The rash got better when the skin cream was used.
• Crime increases: There was more crime with the ban on concealed weapons.
• Crime decreases: The ban on concealed weapons reduced crime.

Table 3

The top graph presents the results for the skin cream questions. On the vertical dimension, the higher the line, the more people answered the question correctly. Horizontally, as you go from left to right, numeracy increases. The four groups all show almost the same results. As you would expect with content-neutral problems, as numeracy increases, more people answer correctly.

The bottom graph shows the results for the gun control questions. There is a tendency for scores to improve with numeracy, but the results are not uniform. The most striking trend is evidence of confirmatory bias. Participants are more likely to give the correct answer in the two conditions where it is consistent with their political views—the liberals when crime decreases and the conservatives when crime increases. When the correct conclusion is inconsistent with the participant's partisan ideology, the lines are almost flat, which means that the high numeracy people are not successfully correcting their biases. The numerate liberals claim that gun control is effective, even when the evidence shows it is not. The numerate conservatives claim that concealed weapons discourage crime, even when the evidence shows they do not. These labels may help to make the graph more clear.

Table 4

What is going on in the minds of these participants? It's likely that participants in all four groups initially engage in System 1 thinking. They look at the data and immediately believe what their political ideology leads them to expect. For the two groups in which the data are consistent with their prior beliefs, this is no problem. If they use System 2 to do the math, they will become more confident of the correct conclusion. But the two groups in which the data are inconsistent with their prior beliefs are faced with a conflict if they engage in System 2 thought. The results suggest that either they don't bother, or if they do, they manage to reinterpret the data in a way that confirms their expectations.

These data support the identity-protective cognition thesis rather than the science-comprehension thesis. It seems unlikely that additional math and science training would help these folks to draw the correct conclusion when that conclusion conflicts with their ideology. One commentator referred to these results as “the most depressing discovery about the brain, ever.” (Journalists have a depressing tendency to refer to the results of psychological research as discoveries about “the brain,” falsely implying that they are physiologically determined. I would say these results demonstrate the effectiveness of political socialization.) They are consistent with findings showing that giving people information which corrects common misconceptions sometimes backfires and causes them to believe them more strongly. It also supplements Kahan's previous finding that the most scientifically literate Americans are not more convinced that climate change is a serious threat, but are more ideologically polarized than those who are less scientifically literate.

Despite the fact that the authors proposed that such results would raise doubts about the possibility of rational self-government, they do not recommend that we curtail participatory democracy and turn over our more important decisions to Big Brother. Kahan sees the problem as a failure of scientific communication. He proposes that scientific issues be reframed so as to reaffirm the ideological beliefs of those who might otherwise be skeptical. For example, in another article he suggests that global warming be reframed as a technical problem to be address by corporations rather than a political problem to be addressed by government:

They [conservatives] would probably look at the evidence more favorably, however, if made aware that possible responses to climate change include nuclear power and geoengineering. . . Similarily, [liberals] are less likely to reflexively dismiss the safety of nanotechnology if they are made aware of the part that nanotechnology might play in environmental protection, and not just its usefulness in the manufacture of consumer goods.

Here is a video of Kahan discussing the results of his global warming studies and his suggestions for improving scientific communication.

Kahan seems to be suggesting that our ideological disagreements can be solved through clever marketing, but this seems an unlikely solution. For example, any serious attempt to curtail climate change through either nuclear power or geoengineering would encounter strong and well-founded objections from important members of the scientific community, and consensus would quickly disappear.

Before we throw up our hands in despair, Kahan's results need to be replicated with a broader range of issues. However, to be fair, global warming is such an important issue that failure to reach consensus on a solution will make all the other issues which we think we care about seem like the equivalent of rearranging the deck chairs on the Titanic.

However, there are some research findings that seem to offer greater hope of overcoming ideological polarization. I will discuss them in future posts. But first, for those that are interested, an appendix about the differences between liberals and conservatives.

You may also be interested in reading:

Is Democracy Possible?  Part 1

Is Democracy Possible?  Appendix

Climate of Mistrust

The Open and Closed Mind

Motivated Reasoning

Is Democracy Possible? Part 1

Let's not be naïve. Obviously, the political system we have now is more a plutocracy than a democracy. Our government has been captured by corporations and the people who own them. But suppose, by some miracle, we were able to enact effective campaign reform laws and politicians became responsive to public opinion. Are American citizens capable of making rational decisions based on scientific information?

Before we go any further, please take a look at this problem.

Table 1

The problem requires correctly interpreting a contingency table. Even bright college students often get it wrong. When deciding whether the skin cream works, some people only compare the numbers in the top row—the number of people who used the skin cream who get better or worse. But this ignores important information from the control group that didn't take the skin cream. Other people only compare the numbers in the left column—the number of people getting better who took the skin cream or didn't. But this ignores the fact that many more people took the skin cream than didn't. To correctly answer this question, you have to consider all four cells of the table. You must compare the percentage of people who used the skin cream who got better to the percentage of people who didn't use the skin cream who got better. Of the 298 people who used the skin cream, 223, or 75% of them, got better. Of the 128 people who didn't use it, 107, or 84% of them, got better. The correct answer is that the skin cream is ineffective.

Why is this problem so difficult? In part, it's because Dan Kahan and his colleagues—who did the study I'm about to report—put their thumb on the scales. They chose data such that, if you use either of the two shortcuts I mentioned—comparing only the numbers in the top row or the left column—you'll get the wrong answer. Fifty-nine percent of the participants in this study answered the problem incorrectly.

This problem illustrates the difference between System 1 and System 2 thinking, the central metaphor of this blog. According to Daniel Kahnemann, in his book Thinking, Fast and Slow, we have two cognitive systems. System 1 is automatic and effortless. We simply say and do what feels right. The System 1 response to this problem is to say that the skin cream works, because that's what it looks like at first glance. System 2 is deliberate and effortful. It comes into play when we take the time to analyze a situation carefully. It can be used to correct the errors that System 1 is prone to making. In this case, System 2 requires that you do the math.

As difficult as this problem is, it has one thing going for it. We don't have any emotional investment in whether this skin cream works. But what if the issue were one about which we had a prior hypothesis? In this case, System 1 could lead us to a second source of error, confirmatory bias—the tendency the interpret new information in a way that is consistent with our prior beliefs and ideological biases. Compare the problem in Table 1 to these three other problems.

Table 2

Does a city-wide ban on concealed weapons increase or decrease crime? Conservatives would probably expect crime to increase (since people can no longer use their concealed weapons to shoot bad guys), and would be resistant to information suggesting that gun control actually works. Of course, the opposite is true for liberals. Kahan expected political ideology to affect participants' responses to the gun control questions, but not the skin cream questions.

Now let's introduce another variable into the mix—numeracy. (I realize this is getting complicated, but please bear with me. It's worth it.) Numeracy—analogous to literacy—refers to mathematical competence and a tendency to use quantitative reasoning in appropriate ways. You would expect numerate people to do better when making judgments of contingency, at least when evaluating skin creams. But what happens when numerate people encounter data contradict that their political views?

We know that liberals and conservatives clash over scientific issues such as gun control and global warming. Kahan suggests two theories to explain these political impasses. The science-comprehension thesis suggests that people don't have enough training in science and math. With better education, people would be capable of correctly interpreting the results of empirical studies. The identity-protective cognition thesis is more pessimistic. It argues that our ideological polarization is so great that it cancels out even the ability of well-educated people to utilize their math and science skills. If the confirmatory bias trumps numeracy, Kahan believes that this raises serious questions about whether Americans are capable of enlightened self-government. We may have to turn over important decisions to committees of experts. (But who will choose the experts?)

What do you think? The answer provided by Kahan's research comes in Part 2.

You may also be interested in reading:

Is Democracy Possible?  Part 2

Is Democracy Possible?  Appendix

Climate of Mistrust

The Open and Closed Mind

Motivated Reasoning

Deja Vu

Our understanding of what has already happened in Syria is grounded in facts, informed by conscience, and guided by common sense.

                                                                                 John Kerry, 8/26/13

Here are the Secretary of State's remarks:

Mr. Kerry is obviously preparing Americans for an illegal U.S.-led military intervention in Syria by claiming that the Syrian regime is guilty of atrocities. The corporate media have fallen into line and are beating the drums of war. This may be a tough sell, since a Reuters/Ipsos poll conducted last week (August 19-23) found that 60% of Americans oppose intervention in Syria while only 9% support it. Of course, past experience shows that the easiest way to increase support for war is to go to war.

In his statement, Mr. Kerry emphasized that the evidence that chemical weapons have been used is “undeniable” and that this action was a “moral atrocity.” His factual claim is almost certainly true, and his moral judgment is shared by almost everyone, but none of this proves that Syrian President Bashar Assad knowingly carried out the chemical attack. His invitation to Americans to watch videos of the dead victims is a transparent encouragement to be swayed by our emotional responses and ignore the lack of evidence that the Syrian government is responsible.

To what extent is our understanding of what happened in Syria “grounded in facts?” So far, Kerry has provided few facts. Although United Nations representatives are investigating the incident, Kerry explicitly stated that this investigation will only determine whether chemical weapons were used, not who used them. Having defined the U. N. fact-finding mission as irrelevant, he seems to be saying that we will intervene regardless of the contents of their report, and maybe before it is released. Meanwhile, our government is showing disturbing signs of confirmatory bias, behaving as if it were a prosecutor gathering evidence in support of its initial hypothesis, rather than exploring all possible hypotheses.

What “facts” have been presented? Kerry said, “We know that the Syrian regime maintains custody of these weapons. We know that the Syrian regime has the capacity to do this with rockets.” This proves absolutely nothing, since the same can be said about the Syrian opposition, which includes some defectors from the Syrian military, and several other Middle Eastern governments and political groups.

Kerry's other argument is that President Assad has delayed U. N. inspections—which, remember, are irrelevant anyway—and is either destroying evidence or allowing it to dissipate. This argument assumes that Assad's only reason for not wanting foreign investigators inside his country is concealment of evidence. As a thought experiment, imagine how Americans might respond if a committee of representatives of enemy nations demanded entry to the D. C. suburbs to investigate our government's activities.

Kerry promises that he will release evidence of Syria's guilt at some unspecified future time. He didn't say whether this will occur before military action is debated in Congress or the U. N. Security Council—if in fact it is debated by either of those bodies. One commentator referred to this speech as Kerry's “Colin Powell moment,” but it seems that the lesson Kerry has taken from Powell's experience is not to be specific about details that later can be proven false. The central point is that, so far, no one has publicly presented any evidence of who was responsible for the attack.

To what extent is our understanding of what happened in Syria “informed by conscience?” There is no doubt that using poison gas is a terrible actrocity, but I'm not sure that the rest of the world is impressed by moral lectures from a country that saturated Vietnam with Agent Orange (also a chemical weapon), that poisoned Iraq with white phosphorous and depleted uranium. A country that stands by while the Egyptian military we fund and support mows down over a thousand protesters in the streets. It might be a useful exercise to discuss whether it is more heinous to murder women and children with poison gas than to burn them alive in drone attacks. Of course, there is very little news footage of the aftermath of drone attacks, and what there is is usually censored in this country by the corporate media.

To what extent is our understanding of what happened in Syria “guided by common sense?” In fact, Kerry seems to be assuming we have checked our common sense at the door. For months, it has been reported that the Syrian government is winning the war and that the insurgency is collapsing. If so, what sense does it make for Assad to risk foreign intervention by using chemical weapons that are unnecessary for him to succeed anyway?  It seems obvious that the insurgents have much more to gain by encouraging foreign governments to become more involved. (Syria's lack of motivation for the attack is so obvious that even the New York Times published an article filled with speculation about why Assad was behaving so irrationally, but, of course, never questioning the premise that Syria is responsible for the incident.)

Kerry may be appealing to a different kind of “common sense”—the knowledge that if you repeatedly tell Americans that some foreign leader is a brutal dictator, they will believe him to be capable of anything. The easiest way to mobilize support for war is to create a negative image of the intended enemy and accuse him of terrible atrocities. And so the recent past repeats itself in the Middle East.

You may also be interested in reading:

A Stroll Down Memory Lane

A Stroll Down Memory Lane

Here's Hubris: Selling the Iraq War, the documentary that was shown on Rachel Maddow's program Monday night. If you didn't see it, it's about 45 minutes long and well worth watching. In fact, we should all watch a brief clip every morning before we pick up the newspaper.

The film traces the origin of most of the false reports leading up to the war. However, it only mentions in passing the body of information coming from United Nations weapons inspectors that rebutted the claim that Saddam Hussein had "weapons of mass destruction." When I predicted in 2002 that no WMDs would be found, I relied heavily on the testimony of Scott Ritter.

Not surprisingly, they don't take a clear stand on whether the Bush administration fell victim to confirmatory bias or this was a deliberate deception.

While they take a few swipes at the media, including the New York Times, they fail to mention MSNBC's own role in selling the war. On February 25, 2003, MSNBC cancelled its highest rated program, Donahue, due to host Phil Donahue's opposition to the upcoming invasion of Iraq.  A leaked memo stated that Donahue would be a "difficult public face for NBC in a time of war."

The Open and Closed Mind

More than three years after Sarah Palin made the false claim that the Affordable Care Act included “death panels” that would decide whether people would receive life-saving medical treatment, 40% of Americans still believe it. A new experiment by Brendan Nyhan, Jason Reifman and Peter Ubel considers the death panel myth in the context of the perseverance effect, a cognitive error in which people's beliefs about reality persist even though the evidence on which they were based has been discredited.

In one perseverance study, participants tried to distinguish between real and fake suicide notes. In fact, regardless of what they said, half the people were told they were right most of the time (the “success” condition), while the others were usually told they were wrong (the “failure” condition). Later, the participants were debriefed. It was explained that the feedback was false and that they had been randomly assigned to success and failure conditions. Even though they understood the debriefing, the “successful” people still believed they were better at the task than those who had “failed” thought they were.

Perseverance affects people's reactions to scientific studies that support or contradict their beliefs. Another experiment involved students with contrasting attitudes, some who believed that capital punishment was a deterrent to murder and others who rejected this claim. People in both groups read summaries of two fictitious studies, one which supported the deterrence hypothesis and one which didn't. You might think reading mixed evidence would lead people to moderate their beliefs. However, both pro- and anti-deterrence participants became more convinced of their original beliefs—a backfire effect. The two groups' beliefs became more polarized.

Perseverance studies seem to suggest that fact-checking politically biased claims is a losing effort. Recent research suggests that more educated and well-informed people show greater perseverance, since having more information allows them to explain away contradictory findings more easily. The Nyhan study is in that tradition. Here is Dr. Ubel explaining the study.

The chart below shows the results. On the left are the low knowledge participants. Without the correction, those who liked Palin believed the death panel myth, but the rebuttal was effective in disabusing them of this belief. However, the high knowledge Palin supporters believed the death panel myth more with the correction than without it. The correction backfired.

Just a word of caution. I was fully prepared to believe this study; however, the more I thought about it, the more I sympathized with the high information Palin supporters. The rebuttal to Palin's claim read (in its entirely) as follows:

However, non-partisan health care experts have concluded that Palin is wrong. The bill in the House of Representatives would require Medicare to pay for voluntary end-of-life counseling sessions, but there is no panel in any of the health care bills in Congress that judges a person's “level of productivity in society” to determine whether they are “worthy” of health care.

Should a well-informed Republican really be satisfied with this rebuttal? If I were in their shoes, I might want to ask some additional questions. Who are these “non-partisan experts?” The correction addresses Palin's claim about end-of-life counseling, but what about the Patient-Centered Outcomes Research Institute, which is charged with evaluating the effectiveness of medical treatments, and the Medicare Independent Payment Advisory Board, which is supposed to refuse to pay for expensive but ineffective treatments? How much indirect control will they have over the treatment patients actually receive? I personally hope these groups will help to eliminate wasteful procedures, but since they haven't met yet, we don't really know.

I'm not questioning the existence of perseverance, or that it makes researchers' lives a lot more difficult. However, another way to read this study is that well-informed people may require more detailed counterarguments than these authors provided.

Motivated Reasoning

The human understanding when it has once adopted an opinion draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises, or else by some distinction sets aside and rejects, in order than by this great and pernicious predetermination the authority of its former conclusion may remain inviolate.

Francis Bacon

Monday's Pittsburgh Post-Gazette contains two articles about drilling for natural gas in the Marcellus shale by the method known as fracking. One of them reports that the U. S. Chamber of Commerce is beginning a new lobbying effort in order to persuade the citizens of Pennsylvania, Ohio and West Virginia not to annoy the natural gas industry by enacting new taxes or environmental regulations. The Chamber will spend “millions of dollars” on this campaign. (The actual amount is not stated.) Gene Barr, president of Pennsylvania's Chamber of Business and Industry, said, “You can call it advocacy or lobbying, but I use the word education.” Presumably, the campaign will be at least as educational as those slick Range Resources commercials that tell us how much a few of our fellow citizens have profited from natural gas royalties.

It would seem that this campaign is a waste of money here in Pittsburgh, since the state of Pennsylvania is already a wholly-owned subsidiary of the natural gas industry, and has given it the most industry-friendly legislation in the country. I guess if you're drilling for natural gas, you can never be too careful.

Elsewhere in the same newspaper, under “National Briefs,” I found an op-ed masquerading as a news article that might easily have been part of the Chamber of Commerce's new campaign. Under the headline, “Experts dispute fracking critics,” it carefully selects paragraphs from the beginning and near the end of this Associated Press article. The original article is already very pro-industry, using selective citation of research to brand several claims made by opponents of fracking as false. The edited version removes what little balance the original contained by removing the last two paragraphs, which report false statements by the natural gas industry.

What caught my eye was a reference to social psychology in order to justify the claim that fracking opponents have fallen victim to self-delusion. Political scientist Mark Lubell attributes the problem an “an actual psychological process” called motivated reasoning, in which people “insist on believing things that aren't true, in part because of feedback from other people who share their views.” 

As a psychological term, “motivated reasoning” is frustratingly vague. It tells us almost nothing about process. But it is true that, since Leon Festinger published A Theory of Cognitive Dissonance in 1957, social psychologists have conducted thousands of studies showing that people are more likely to accept information that supports their prior beliefs than information that contradicts them. Clearly this is something that we all do a lot of the time. Here's journalist Chris Mooney explaining motivated reasoning and explaining one of the processes involved.

Brendan Nyhan and Jason Reifler did four studies designed to test whether media reports can be expected to correct false beliefs. Participants read two mock news articles—a false claim, followed by an article which corrected it. For example, one study presented President George W. Bush's false claim from after the invasion of Iraq that Saddam Hussein had possessed weapons of mass destruction, followed by an article about the Duelfer Report that refuted that claim. All four studies found evidence of motivated reasoning, in that the corrections did not significantly reduce the original misperceptions in their overall sample.

More importantly, two of the studies found evidence of a “backfire effect,” in which the correction increased belief in the original false claim, among certain subgroups. Subjects were asked to classify themselves on a 7-point scale of liberalism-conservatism. In the WMD study described above, the liberals and centrists either corrected their false beliefs or were unchanged, but the conservatives became more extreme in their belief that WMDs had been found. However, this backfire effect failed to occur when the study was repeated.

The backfire effect was replicated, however, in a second study in which participants were first told that tax cuts would stimulate the economy and increase government revenue, and then told that research had shown this claim to be false. Again, it was the conservatives who increased their belief in the efficacy of tax cuts after being shown studies that found them to be ineffective. The fourth study was conducted in order to try to find a backfire effect among liberals. Subjects were falsely told that President Bush had completely banned stem cell research, and later correctly told that he had only banned federally-funding research, but allowed privately-funded research to continue. In this case, participants once again persevered in their original beliefs, but no backfire effect was found.

Here's Brendan Nyhan discussing this research and making some suggestions as to how journalists can correct their audience's false beliefs.

The Nyhan-Reifler studies seem to suggest that conservatives are more likely to engage in motivated reasoning than liberals. In fact, after reviewing the entire literature on this subject, Chris Mooney comes to this conclusion. However, we need to be extremely skeptical before accepting this hypothesis. It's certainly possible that liberals would show a backfire effect if the study involved an issue that was more central to their political ideology.

Here's the problem with the fracking article. It's far too easy to read the motivated reasoning literature selectively and use the concept as a weapon to attack your political opponents, thereby engaging in motivated reasoning about motivated reasoning. The claim of motivated reasoning should only be used when the argument someone is defending has been clearly refuted by the evidence. The AP article fails to distinguish between claims that are false and those that are merely unsubstantiated. A claim can be unsubstantiated if it has not been conclusively tested, or if the available evidence is mixed.

While it is certainly true that some opponents of fracking have made false or irresponsible accusations, the claims that fracking will contaminate the air or ground water, or that it will expose people to cancer-causing chemicals, are plausible and serious enough to justify independent—that is, not industry-funded—research at multiple locations. The argument that natural gas is a cleaner burning fuel than coal has been questioned by highly competent research. This debate is further complicated by the fact that it may take decades for conclusive evidence of some harms caused by fracking to emerge.

Meanwhile, we should not be surprised if our corporate media, which are heavily dependent on advertising by the fossil fuel companies, treat industry propanganda as if it were unbiased and accuse those who question it of motivated reasoning.