Research Highlights Article
April 24, 2024
Information inequality
Do US voters share a common baseline of political facts?
Source: Melissa Kopka
In recent years, fake news, disinformation, and social media have put a spotlight on the need for an informed electorate in modern democracies. But while some observers place the blame for “post-truth” politics on partisan echo chambers, the biggest gaps in voters' knowledge may come from socioeconomic factors, according to a paper in the American Economic Review.
The authors, Charles Angelucci and Andrea Prat, investigated how well a cross-section of US voters were able to sift true and false news stories, as assessed by mainstream journalists.
“We felt like this huge focus on fake news, even though it makes sense, came at the expense of our understanding of how information about real news is distributed in society,” Angelucci told the AEA in an interview.
In order to measure Americans' knowledge of current events, however, a tricky question had to be answered first: What political news should voters be informed about?
One common approach is for researchers to survey Americans by designing questions about political news themselves. But that method is in danger of being skewed by the subjectivity of the researchers, according to Angelucci.
Even though we agree on what's true and what's false, we draw completely different conclusions from that. In a sense, that's a very pessimistic conclusion from our paper, because that's much harder to fix. How can you change the way people reason, and should you even try to?
Charles Angelucci
He and Prat controlled for this potential bias by outsourcing their questions to a panel of journalists from prominent news outlets. First, they asked these mainstream journalists to select headlines from Reuters that they felt were the most important news of the week. Second, they asked them to write a set of fake, but seemingly believable, headlines in the same style as the real news.
“That doesn't make the choice of the questions or the news any less subjective, but at least it's the subjectivity of the quote, experts, unquote and not the researchers,” Angelucci said.
With these journalistically true and false headlines in hand, the authors quizzed, on a monthly basis, roughly 15,000 US voters over 11 months. Out of six headlines, the respondents were asked to identify which three were true and which three were false. In return for the correct answer, they were given a financial reward.
However, even with the help of the journalists, there was still a concern that the authors' questions did not accurately measure voters’ ability to discern fact from fiction. Education research demonstrates that test results can be just as much a function of test design as the knowledge of test takers.
In the context of the authors’ study, their survey results could potentially be influenced by either the plausibility or the partisan slant of the fake headlines. Using a nominal response model to account for these factors, the authors estimated the ability of respondents to discern journalistic truths, as well as the role of partisan bias, in their answers.
They found that partisanship played some role in the headlines selected. When a headline reflected favorably on a respondent’s preferred political party rather than unfavorably, they were two percentage points more likely to select a true news story.
However, age, education, gender, income, and ethnicity each contributed more to the probability that a respondent selected a true story than partisan affiliation. For each of these socioeconomic characteristics, the researchers found a 4-6 percentage point difference in the subjects' ability to select the true story. In particular, older, high-income, college-educated, White men were the most likely to pick out the real headlines. The probability that such individuals identified a true story was 18 percentage points greater than their counterparts with the opposite characteristics—nearly an order of magnitude larger than the partisan effect.
Breaking down information inequality
In the chart below, Panel A shows the probability that individuals belonging to various socioeconomic groups selected the true news story. Panel B reports the probability that individuals assigned a three-to-one or higher odds of truth in favor of the true news story. The dashed vertical lines correspond to the average overall.
Overall, the authors found that, on average, 82 percent of US voters will correctly choose a true news story over a fake news story. Moreover, nearly half will confidently pick the true story, while only 3 percent will confidently pick the false story.
The results highlight that most US voters are able to identify basic facts about current events regardless of partisan leanings. But it also suggests that those facts are interpreted differently.
“Even though we agree on what's true and what's false, we draw completely different conclusions from that,” Angelucci said. “In a sense, that's a very pessimistic conclusion from our paper, because that's much harder to fix. How can you change the way people reason, and should you even try to?”
♦
“Is Journalistic Truth Dead? Measuring How Informed Voters Are about Political News” appears in the April 2024 issue of the American Economic Review.