The Neural Network That Lies: The Root Causes of AI Bias

We are used to thinking of a computer as the epitome of impartiality. In reality, AI only knows which word is most likely to appear next in order to minimize prediction error. The pursuit of minimizing errors turns harmless statistics into a weapon of systemic bias. In recent years, we have witnessed dozens of scandals where AI exhibited prejudice against people based on gender, race, age, or location. But bias is only half the story. The real danger lies in the fact that neural networks not only reflect but actively amplify existing stereotypes. Responses also depend on the model’s temperature, the time of day, and a range of other parameters. How did this happen?

An Inconvenient Legacy

A 2021 study by a Stanford research group showed that in the Common Crawl corpus (which was included in The Pile dataset to train GPT-3), the ratio of “he” to “she” mentions in connection with professions like “engineer,” “scientist,” and “judge” was approximately 7:1 in favor of the masculine pronoun. For professions like “nurse,” “teacher,” and “housewife,” the ratio was reversed. It would seem that if a model simply reproduced these proportions, querying it about a programmer should yield “he” 70% of the time and “she” 30%. In reality, however, the disproportion can reach 95:5. Why? The reason is that the model is trained not just to memorize frequencies, but to predict the most probable continuation within a specific context.

When the context is neutral (with no explicit indication of gender), the model relies on preset probabilities and utilizes an attention mechanism that links the words “programmer” and “he” through a multitude of indirect cues: verbs, adjectives, and semantic surroundings. As a result, even a minor skew in the data is multiplied by the depth of the neural network, making the final output shift significantly larger than the initial input bias.

Stereotype amplification is a phenomenon where the model doesn’t just repeat the frequency distribution of features found in the training data; it shifts the final distribution toward an even more polarized version. A prime example is the skew in evaluating the age of men and women regarding competence, as discussed in a 2025 Nature article by Douglas Guilbeault and colleagues titled, “Age and gender distortion in online media and large language models.”

Mathematically, this occurs because the model can artificially inflate logits (mathematical estimates) due to overconfidence and a desire to play it safe. The model “prefers” to err on the side of the majority because, when minimizing the Cross-Entropy Loss function, the algorithm incurs a massive penalty for failing to guess the dominant class. Therefore, it “trims” the tails of the distribution, rendering the world far more stereotypical than it actually is.

How AI Penalties Work — An AI penalty is a mathematical rule that issues penalty points to an algorithm when it makes a mistake. The algorithm’s ultimate goal is to accumulate as few of these points as possible. For instance, an AI navigator receives no penalty for finding the shortest route, but gets 1 point for every extra kilometer and 5 points for every minute spent in traffic. This trial-and-error points system is how AI learns to choose the optimal solution.

Trust, But Verify

Researchers have developed several validated methods to quantify the level of gender bias in Large Language Models (LLMs). One of the most famous tests is WinoBias, introduced in 2018 by a team from UCLA (University of California, Los Angeles). It consists of thousands of syntactically ambiguous sentences where the pronoun “he” or “she” could refer to one of two characters. WinoBias tests how the model resolves such anaphoric links.

However, with the evolution of Reinforcement Learning from Human Feedback (RLHF) and the introduction of strict censorship filters, developers of modern models (like GPT-4, Gemini, and Claude) have learned to “coach” models to pass WinoBias. Today, they often pass the test with a high accuracy that rivals human performance. And here, a new problem arises: overcorrection. In the pursuit of political correctness, a model might systematically begin assigning high-status professions to women, even if it explicitly contradicts the context. A glaring example of this semantic failure was the major international scandal in February 2024 involving Google’s Gemini neural network. Heavily tuned by developers for mandatory racial diversity, the model began generating images of Black people as US Founding Fathers and WWII German Wehrmacht soldiers. This is an artificial distortion in the opposite direction, which is equally harmful because it creates a false representation of reality.

A more reliable and nuanced tool is the counterfactual method. The concept is straightforward: take a large dataset of real texts containing gender markers. Then, create two identical copies of the corpus, changing only male names and pronouns to female ones (and vice versa) in one of them, while leaving the rest of the content untouched. Afterward, run both datasets through the model to perform a task: evaluate a qualification, make a medical diagnosis, or assign a credit score. The discrepancy in results between the two versions serves as direct evidence of pure gender bias, isolated from all other factors.

A third approach is calculating the distances between word vector representations within the model’s spatial architecture. This method does not require text generation; instead, it analyzes the fundamental structure of the trained neural network. Researchers take word pairs — “man-woman,” “programmer-nurse,” “manager-subordinate” — and measure the cosine distance between the corresponding vectors. If the distance between the “programmer” and “man” vectors is shorter than between “programmer” and “woman,” it stands as mathematical proof of an associative link.

In Tolga Bolukbasi’s famous 2016 paper, it was shown that in word2vec embeddings — which form the foundation of many models — stereotypes are so pronounced that they can literally be “subtracted.” For example, executing the operation “programmer – man + woman” results in a vector closely resembling “housewife.” Model architectures have grown far more complex since then, but the fundamental property remains: because training occurs on texts where correlations between gender and profession objectively exist (even if only for historical reasons), the neural network inevitably encodes them. Even if a model is trained from scratch on the same datasets, the biases will reproduce themselves.

AI Hypnosis in the Job Market

Until now, we have discussed abstract tests and statistical skews. But algorithmic bias has a tangible impact on the lives of millions. Let’s examine two highly sensitive areas where text-based neural networks are already integrated into daily practice: recruiting and healthcare. Large corporations worldwide use LLMs for the initial screening of job candidates. A recruiter uploads 500–1,000 resumes into the system and tasks the model: “Write a short summary for each candidate, highlight key skills, and assess leadership potential.” The model only processes text and, on the surface, appears objective.

However, as we have established, resume text is rife with implicit gender and age markers: the use of active verbs, mentions of hobbies and volunteer work, graduation years, and lengths of tenure. The model can still “guess” gender and age from these clues, comparing the candidate against “typical” patterns. Consequently, AI widens the career advancement gap instead of closing it.

In 2018, Amazon scrapped its hiring system after discovering it discriminated against women, but many companies continue to deploy opaque AI systems without proper auditing, buying into the myth of machine objectivity. And when, for example, a professional over 40 is denied an interview, they may never know that the real reason wasn’t a lack of qualifications, but an algorithmic age bias. With proper tuning, AI can uncover hidden talent and potential, but to achieve this, the goal of recruiters must shift from merely filtering resumes to conscious, professional scouting.

In healthcare, the stakes are even higher: AI diagnostic assistants are rapidly gaining traction, particularly in cardiology. However, a 2024 study published in the German Journal of Cardiology showed that models were significantly more likely to recommend invasive procedures and ECGs for men, while classifying identical complaints from women as non-cardiac. This is driven by poor input data quality: the sensitivity of a standard ECG in women is only about 68%, leading to false negatives in 36% of cases, compared to less than 10% in men. By learning from these errors, the algorithm elevates medical bias to an absolute rule — and if a doctor trusts the algorithm, they risk missing a real disease.

(Not So) Magic Pill

It is impossible to completely eliminate algorithmic bias in LLMs because it is rooted deeply in the very structure of human language and data. We can only mitigate its impact, for example, by using model ensembles (where one searches for signs of discrimination while another makes the decision) or by implementing confidence thresholds where the system escalates disputed cases to a human.

However, fact-checking remains the primary tool for combating AI imbalance. Every organization planning to use LLMs for decisions that affect people must conduct an independent fact-checking audit for gender, racial, and age bias. This must include counterfactual tests, vector analysis, and rigorous checks for overcorrection. Development institutions need to train people to critically evaluate AI prompts. On a national level, standards must be adopted that obligate developers and users of algorithms to warn of potential bias and provide avenues for humans to appeal AI-made decisions.