How content analysis helps to expose fakes

Fact-checking is not just about checking individual statements, but also about analyzing large amounts of data. Content analysis helps organize information, find patterns, and identify manipulations in texts and media.

Content analysis is a method of systematically studying the content of text and graphic data to identify key themes, trends, and patterns. This method allows researchers to analyze large amounts of information and identify hidden patterns, making it a useful tool in a variety of fields, including communications research, sociology, political science, medicine, and, of course, fact-checking.

The main stages of content analysis

  • Defining objectives and calculations. Researchers first define the objectives of the study, formulate research questions, and develop an analysis plan.
  • Data collection. Content analysis requires collecting data to be analyzed. This may be texts from various sources, including printed media, the Internet, social networks, audio and video recordings, images, and others.
  • Coding. At this stage, researchers develop a coding system that allows them to structure and classify the data according to the objectives of the study. Coding can be qualitative (descriptive) or quantitative (assigning numerical values).
  • Analysis and interpretation. After coding, researchers analyze the coded data to identify trends, thematic areas, frequently occurring concepts, and other important aspects. Interpretation of the results allows for drawing conclusions and answering the research questions.
  • Checking reliability and validity. An important stage of content analysis is checking the reliability and validity of the results. This may include intercoder agreement (consistency of results between independent researchers), as well as checking for compliance with the methodological requirements of the study.

Application of content analysis

  • Media analysis. Content analysis is used to study media content, such as news, articles, advertising, to identify thematic trends, propaganda, bias and other aspects:

– Studying narratives in fake articles (e.g. identifying recurring clichés: “scientists are hiding”, “authorities are hushing up”).

– Comparing the tone in reliable and fake materials (e.g. more emotional assessments in fakes).

– Content analysis of headlines of different media for bias (e.g. how one event is presented in pro-government and opposition publications).

– Identifying the frequency of mentions of certain persons or topics (e.g. hushing up some events and exaggerating others).

Example: In 2020, an analysis of nearly 200 Facebook* posts in the US that made false claims ahead of the Georgia runoff election found that 60% of them had not been flagged by Facebook’s fact-checking partners. This demonstrates how platforms failed to label misleading content about the election.

  • Detecting bots and coordinated social media campaigns

– Analysis of social media posts for template phrases, publication time, and duplicate accounts.

– Search for patterns in comments (e.g., mass use of identical lines).

– Analysis of the spread of disinformation in thematic communities.

– Identification of key sources of false narratives (bots, public networks, fake experts).

Example: A study of data on Social Media X (formerly Twitter) around the 2019 UK election used content analysis to measure propaganda, identify coordinated communities, and the use of bots amplifying political narratives.

  • Political analysis. Content analysis is used to study political documents, speeches, appeals to identify political trends, strategies and rhetoric.

– Analysis of the frequency of use of certain terms in politicians’ speeches (e.g., “national security”, “threat”).

– Analysis of public speeches for false statistical data.

– Identification of manipulative techniques (e.g., substitution of concepts, false analogies).

– Comparison of promises in election programs with real actions.

Example: In a randomized field experiment conducted in 2023, researchers analyzed the statements of 55 Italian MPs over 16 weeks, starting in March 2021. Politicians whose statements were fact-checked significantly reduced their number of false claims — and maintained this effect for at least eight weeks. This example shows how content analysis not only identifies false statements but can also influence rhetoric, changing the nature of political discourse.

  • Sociological research. In sociology, content analysis allows us to study images, ideas and stereotypes present in public discussions and media reports.

– Analysis of the tone of responses about social groups (migrants, minorities) to identify negative/positive bias.

– Checking the frequency of mentions of problems (for example, comparing coverage of environmental and economic topics).

– Analysis of question wording (identifying leading or ambiguous wording that may distort the results).

– Comparison of data from different surveys on the same topic to identify possible contradictions or manipulations.

Example: In the UK, more than 63 million words were analysed from 52,990 news articles and 317 House of Commons debates on immigration between 2019 and the July 2024 general election. The researchers found that the media’s and politicians’ use of language around race and immigration had contributed to “an increase in reactionary politics and a backlash against anti-racism, emboldening the far right in this country.”

  • Marketing research. In marketing, content analysis is used to analyze advertising materials, consumer reviews, social media to identify consumer preferences and trends.

– Comparison of marketing strategies (e.g. frequency of use of certain words in advertising).

– Research of the competitive environment (e.g. analysis of the number and tone of brand mentions in the media and social networks).

– Analysis of consumer sentiment

– Checking advertising claims and comparing them with scientific data.

– Detecting hidden advertising in blogs and the media.

– Detecting fake reviews and cheating (mass reviews over a short period, template phrases, unnatural wording)

– Identifying real customer problems (analysis of complaints vs. official company statements).

Example: Following the 2020 US election, where Joe Biden won and Donald Trump disputed the results, researchers looked at how people discussed the vote on social media. They analyzed tweets before, during, and after the election to see if users’ sentiments matched the actual results.

Challenges and Prospects

Content analysis, despite its effectiveness, faces a number of challenges, such as subjectivity in data coding, the complexity of processing large amounts of information, and the need to use modern big data analysis methods. However, with the development of technologies and research methods, content analysis remains an important tool for understanding the context and analyzing text and graphic data.

Using content analysis in fact-checking helps not only to find fakes, but also to understand how false information is spread. The main thing is to follow a clear methodology and a critical approach to interpreting data.