AI for Fact-Checking: A User Manual

Artificial intelligence has radically transformed the fact-checking process by offloading the heavy lifting of data management. However, its primary paradox lies in the fact that while it saves us time, it demands more of our attention: the ability to critically evaluate its responses is now more important than the ability to generate them. This article serves as a practical guide to building a dialogue where AI acts not as an oracle, but as a powerful assistant that requires constant oversight.

Modern artificial intelligence (AI) tools have significantly reshaped the approach to fact-checking. Today, neural networks handle much of the routine work: they help process massive datasets quickly, cross-reference disparate sources, and identify non-obvious patterns within information. We have previously written about how AI assistants automate processes across various fields and how to leverage their capabilities effectively, including for fact-checking.

This type of delegation allows researchers to focus on analysis and meaning rather than the technicalities of searching. However, these same capabilities demand a new level of caution. AI systems, while adept at finding connections, can sometimes create false ones — generating convincing but non-existent quotes, distorting context, or subtly shifting emphasis. It is therefore vital to remember that AI is a tool in the hands of a specialist, not a fully autonomous system capable of replacing human critical thinking.

In essence, the key skill of the modern fact-checker has shifted from simple information retrieval to the ability to maintain a reliable dialogue with AI. This dialogue is built on a clear division of roles: the neural network is the assistant that proposes hypotheses, gathers data, and assists with analysis, while the human acts as the rigorous editor and final arbiter, tasked with questioning and double-checking every response.

This article provides a step-by-step guide to establishing such a workflow. We will explore how to properly assign tasks to AI at each stage of verification, how to recognize and bypass typical algorithmic errors, and how to maintain control over the final conclusion.

A Step-by-Step Guide: Building an AI Dialogue for Fact-Checking

Step 1: Deconstructing the Claim into Verifiable Elements

The Human’s Role:

  • Identify the core claim requiring verification.
  • Clearly define all elements to be checked: dates, figures, names, and cause-and-effect relationships.
  • Formulate verification questions for each element.

How AI Can Help: AI can automatically analyze the text of a claim, recognize and categorize its key components (dates, names, terminology), and structurally break down a complex sentence into simpler, more manageable parts. Based on this analysis, AI will generate a list of specific verification questions for each component and suggest interpretations for ambiguous phrasing that requires clarification.

Example of a correct prompt:

“Break down the claim ‘Since February 2022, the price of gas in Europe has increased tenfold’ into individual verifiable components. For each component, formulate a specific verification question.”

Example of AI response and human actions:

AI identifies:

1. Timeframe: “Since February 2022” → Question: “What specific period: February 2022 to the current date, or February 2022 to the peak value?”

2. Geography: “In Europe” → Question: “Which specific countries or markets are meant: TTF (Netherlands), NBP (UK), or the EU average?”

3. Commodity: “Gas” → Question: “What type of gas: spot prices, futures, or physical deliveries?”

4. Change: “Increased tenfold” → Question: “From what baseline is the growth calculated: February 2022 prices or pre-crisis levels?”

Human actions after receiving the response:

  • Clarifies ambiguities (e.g., “Europe” → “EU + UK”).
  • Finalizes the list of verification tasks.
  • Saves this list for the subsequent workflow.

Step 2: Sourcing Authoritative Data

The Human’s Role:

  • Determine which organizations are the primary sources for this type of data.
  • Compile a list of official and reputable sources.
  • Verify the reputation and methodology of these sources.

How AI Can Help: Drawing on its internal knowledge base, AI can suggest a list of potentially relevant sources: official bodies, scientific databases, industry reports, and statistical aggregators. It can describe their typical methodology and strengths, and generate optimized search queries (including in multiple languages) to quickly find the necessary pages or documents on these organizations’ websites.

Example of a correct prompt:

“To verify gas prices in Europe since 2022:

1. List official data sources (exchanges, regulators, agencies).

2. Indicate their strengths and weaknesses.

3. Provide specific search queries in English and Russian for each source.”

Example of AI response and human actions:

AI suggests:

  • Source: TTF (Title Transfer Facility) — primary Dutch gas futures.
  • Query: “TTF gas futures historical prices 2022-2024”
  • Source: Eurostat — official EU statistics.
  • Query: “Eurostat natural gas prices for household consumers”
  • Source: IEA (International Energy Agency).
  • Query: “IEA European gas market report 2023”

Human actions after receiving the response:

  • Verifies the relevance of the sources (visits the websites).
  • Compares the methodologies of different sources.
  • Selects the 2–3 most relevant sources for in-depth analysis.
  • Critically verifies each AI-suggested source for the existence and availability of data.

Step 3: Data Extraction and Analysis

The Human’s Role:

  • Collect data from the selected sources.
  • Ensure data comparability (consistent timeframes, units of measurement).
  • Identify potential discrepancies between sources.

How AI Can Help: When provided with structured data (tables, CSV, PDF), AI can perform calculations, create comparative tables, and calculate key metrics (percentages, ratios, trends). It can scan large numerical datasets for statistical anomalies or outliers that require extra attention and visualize primary patterns to aid comprehension.

Example of a correct prompt when uploading a data file:

“The uploaded file contains monthly TTF gas prices from 2020 to 2024:

1. Calculate how many times the price in each month of 2023–2024 differs from the price in February 2022.

2. Highlight the months with the maximum and minimum growth.

3. Note any anomalous values that require additional verification.”

Example of AI response and human actions:

AI provides:

  • A table of growth coefficients: August 2022 – 8.7x increase from February 2022; current month – 2.3x increase.
  • Highlights values that appear statistically atypical.

Human actions after receiving the response:

  • Double-checks calculations for key months.
  • Cross-references highlighted “anomalies” with the news context of that period.
  • Formulates an interim conclusion: “The claim of a tenfold increase refers to the peak value in August 2022 but does not reflect the current situation.”

Step 4: Contextualization and Benchmarking

The Human’s Role:

  • Place the data within a historical and comparative context.
  • Assess whether the cited change is unique or typical.
  • Account for seasonal factors and long-term trends.

How AI Can Help: AI can quickly scan its knowledge base or provided documents to find historical analogs and periods with similar dynamics. It can draw parallels between events, highlighting commonalities and differences, and suggest a set of factors (economic, political, social) for an in-depth comparative analysis.

Example of a correct prompt:

“Based on TTF gas price data:

1. Compare the price growth in 2022–2023 with previous price shocks.

2. Find similar periods of sharp energy price increases over the last 20 years.

3. Suggest factors for comparison: duration of the shock, recovery speed, global context.”

Example of AI response and human actions:

AI notes:

  • 2005-2008: Gradual 300% increase over 3 years.
  • 2022: 800% increase over 6 months.
  • Contextual differences: Geopolitical triggers vs. market imbalances.

Human actions after receiving the response:

  • Verifies every historical comparison against alternative, reliable sources.
  • Asks the AI for clarification: “What specific events caused the growth in 2005–2008? Provide dates and descriptions.”
  • Builds an understanding of the current situation’s uniqueness.

Step 5: Drafting a Balanced Verdict

The Human’s Role:

  • Synthesize all gathered data into a coherent conclusion.
  • Clearly state the degree to which the claim is confirmed or debunked.
  • Add necessary nuances and research limitations.

How AI Can Help: AI can serve as an editorial assistant: based on raw data and interim notes, it can generate a structured draft of the conclusion. It can suggest a clear verdict (Confirmed / Debunked / Partially Confirmed), accurately integrate specific figures and facts into the text, and add standard sections regarding sources and analysis limitations, following a specified template.

Example of a correct prompt:

“Based on our data, generate a draft conclusion for the claim: ‘Since February 2022, the price of gas in Europe has increased tenfold.’ Structure:

1. Main verdict (Confirmed / Partially Confirmed / Debunked).

2. Specific figures with timeframes.

3. Important nuances and context.

4. Data sources.

5. Limitations of our analysis.”

Example of AI response and human actions:

AI suggests a draft: “The claim is partially confirmed: In August 2022, TTF gas prices peaked at 8.7 times the February level but did not reach a tenfold increase. As of now, the increase stands at 2.3 times. Important context:…”

Human actions after receiving the response:

  • Checks the AI text for “hallucinations” or inaccuracies.
  • Edits the draft, adding professional assessment.
  • Ensures all nuances are addressed and the conclusion is understandable to a layperson.
  • Finalizes the version with the label “Human-Verified.”

By mastering this algorithm, you transform artificial intelligence from a source of questionable answers into a structured professional tool. However, the process alone does not guarantee accuracy — it is vital to understand its weak points. In our next piece, we will examine common AI Pitfalls in Fact-Checking so you know exactly what to look for when analyzing its responses.