AI is Not a Legal Search Engine: Why Chatbots Invent Court Cases (And How to Catch Them)

Many professionals — especially in law and research — are treating generative AI like the next evolution of traditional databases. They assume that when they type a query into a chatbot, the system searches an objective index of facts and returns the best result.

This is a dangerous misconception. An AI is not a search engine; it is an overly eager, people-pleasing intern.

If you understand this fundamental difference, AI can be a handy productivity booster. If you don’t, it might just cost you your career. As a practicing lawyer, my own experience with AI has swung drastically between these two extremes.

The 5-Minute Courtroom Miracle

A redacted court document confirming Dr. Alexandre Guerreiro’s participation as defense counsel in the trial.

An extremely positive example of AI use occurred during a mega-trial at the Tribunal da Comarca da Guarda (Central Criminal Court) in Portugal in January 2026. The trial involved a collective court, seven defendants, and a massive case file of exactly 2,015 pages.

During the afternoon session, the presiding judge made a surprise decision to hear testimony from a person named M., who had never been scheduled to testify. Understanding all of this witness’s interventions in the proceedings would have been impossible for a human in mere minutes. Given the surprise effect, we asked the judge for a short, 5-minute break to gather information.

Because of strict data privacy rules and attorney-client privilege, my team exclusively uses local AI applications (like LM Studio and Qwen) running directly on our laptops to process documents with sensitive data, ensuring no internet connection is used. I asked our local AI to search the digitized file by the witness’s name and related facts.

In seconds, the AI showed us all the pages where M. appeared and mapped out connections to other facts that the human eye would not easily perceive in such a short window.

The judge’s brief recess gave us just enough time to confirm the relevance of the AI’s findings. It was a massive victory for AI efficiency.

The Contrarian Reality: Conversational Sycophancy

Emboldened by successes like this, it is easy to start relaxing and trusting what AI provides. As a lawyer, it is very useful to cite decisions from higher courts to reinforce a position. Traditional legal search engines require exact keyword matches, which can lead to hours of fruitless searching. AI, intuitively understanding natural language, seems like the perfect solution.

However, traditional legal databases (like LexisNexis) operate on retrieval. If a case doesn’t exist, you get a blank page. Large Language Models (LLMs), on the other hand, operate on prediction. They are fine-tuned for “conversational sycophancy”—meaning they are inherently biased toward giving you a satisfying, helpful-sounding answer.

If a lawyer asks an AI to find a higher court ruling that supports a highly specific, niche legal argument, the AI will try its hardest to help. And if that case doesn’t exist? It can invent a beautifully formatted, entirely fake ruling. In the AI industry, this is called a hallucination.

To demonstrate this behavior, I recently ran a test using Gemini. I asked the AI for jurisprudence stating that it is allowed for a victim of domestic violence to record an argument with their companion. It successfully delivered valid citations. But when I asked for jurisprudence supporting the exact opposite—that such recordings are not valid evidence—it immediately failed.

When I confronted the AI because I couldn’t find the specific appellate court ruling it cited, its response was chillingly casual:

The AI explicitly admits to a “hallucination” after being confronted about a fabricated appellate court ruling.

Notice how the AI openly admits to “hallucinating” in an attempt to force a result that satisfied my prompt. Even worse, to compensate for its failure and satisfy my demand, it immediately pivoted and confidently offered me three completely fabricated decisions:

In an attempt to “compensate” for the fake citation, the AI immediately generates three new cases, demonstrating the danger of conversational sycophancy.

So, how can legal professionals and researchers harness the incredible efficiency of AI without falling victim to its hallucinations? The answer lies in strict verification protocols.

Read the follow-up guide by GFCN and Alexandre Guerreiro: “How to Safely Use AI in Legal Practice” to learn the exact step-by-step rules for protecting your workflow.