Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://vlmsarebiased.github.io/ below:

VLMs are Biased

^*Equal contribution ^†Equal advising
¹KAIST, ²College of William and Mary, ³University of Alberta, ⁴Auburn University

Finding: State-of-the-art Vision Language Models achieve 100% accuracy counting on images of popular subjects (e.g. knowing that the Adidas logo has 3 stripes and a dog has 4 legs) but are only ~17% accurate in counting in counterfactual images (e.g. counting stripes in a 4-striped Adidas-like logo or counting legs in a 5-legged dog).

VLMs don't actually "see" - they rely on memorized knowledge instead of visual analysis due to bias.

Figure 1: VLM Failures Overview

VLMs fail on 6 counting tasks and one low-level vision task across seven domains

VLMs fail on 6 counting tasks (a–e & g) and one low-level vision task (f). State-of-the-art models achieve perfect performance on original images but catastrophically fail when objects are subtly modified, defaulting to memorized knowledge rather than actual visual analysis.

Imagine asking GPT-4o to count the legs of an animal, and it gets it right every time. Impressive, right? Now imagine adding just one extra leg to that animal and asking again. Suddenly, it fails completely.

Original dog (4 legs): All models get it right
Same dog with 5 legs: All models still say "4"

They're not counting - they're just recalling "dogs have 4 legs" from their training data.

Figure 3: Subtle Modification Failures

VLMs fail to detect subtle changes in counterfactuals and default to biased answers

VLMs fail to detect subtle changes in counterfactuals (CF) and default to biased answers. Despite clear visual modifications (extra legs, extra stripes), all models consistently output the expected "normal" values rather than counting what they actually see.

The Core Issue: VLMs suffer from severe confirmation bias. When they see familiar objects, they default to memorized knowledge instead of performing actual visual analysis. This isn't a minor glitch - it's a fundamental flaw in how these models process visual information.

Our testing methodology follows a simple but powerful three-step process that exposes the fundamental difference between memorization and actual visual analysis in VLMs.

Figure 2: VLMBias Testing Framework

Three-step methodology: (a) Confirm VLM knowledge, (b) Test on counterfactuals, (c) Analyze bias-relevant backgrounds

Given a subject (e.g., Adidas logo), we first confirm that all VLMs have sufficient knowledge about the subject via ID and counting sanity-check questions (a). Then, we test VLMs on the counterfactual image (b) and report accuracy on counting (Q1 & Q2) and Y/N identification tasks (Q3). For all tasks, we test the hypothesis that visual bias cues in the background (c) may be so strong that they cause VLMs to ignore the modified object and default to biased answers.

Confirm VLMs have the knowledge

• ID Question: "What shoe logo is this?" → "Adidas" ✓
• Counting Question: "How many stripes?" → "3" ✓

Result: 100% accuracy on original images across all models

Test on counterfactual images

• Q1: "How many visible stripes?" → "3" ✗ (should be "4")
• Q2: "Count the visible stripes" → "3" ✗ (should be "4")
• Q3: "Is this the Adidas logo?" → "Yes" ✗ (should be "No")

Result: 17.05% average accuracy - catastrophic failure!

The Critical Insight: The gap between Step 1 (100% accuracy) and Step 2 (17% accuracy) proves that VLMs are not actually "seeing" - they're retrieving memorized associations. When the visual evidence contradicts their training data, they consistently choose memorized knowledge over what's actually in the image.

Explore examples from all 7 domains where state-of-the-art VLMs fail spectacularly.

Task 1: Animals

Modified Animal Images - Adding extra legs to test counting ability

Animals with Extra Legs

Models consistently say "2 legs" for 3-legged birds and "4 legs" for 5-legged mammals.

Animals

Key Finding: Worst performance domain. Models defaulted to canonical leg counts even when modifications were clearly visible and anatomically plausible.

Task 2a: Shoe Brand Logos

Modified shoe logos with extra stripes and curves

Modified Shoe Logos

Models default to canonical brand specifications even when logos are clearly modified.

Shoe Logos

Key Finding: Models defaulted to canonical brand specifications. Even when logos were clearly modified and placed in realistic sports contexts, VLMs stuck to memorized brand knowledge.

Task 2b: Car Brand Logos

Modified car logos with extra circles and star points

Modified Car Logos

Car logos appear smaller making VLMs even more reliant on brand memory.

Car Logos

Key Finding: Worst performance in logos category. Small logo size relative to the vehicle made visual bias even stronger - models completely ignored modifications.

Task 3: National Flags

Modified flags with extra or missing stars and stripes

Modified National Flags

Models memorized flag facts rather than counting visible elements.

National Flags

Key Finding: Better performance on star counting (11.79%) than stripe counting (4.52%). Stars are spatially separate while stripes are adjacent, making stripe modifications harder to detect.

Task 4: Chess Pieces

Chess boards with modified piece counts

Modified Chess Starting Position

Models defaulted to standard 32-piece count despite pieces being missing.

Chess Pieces

Key Finding: Best performance counting task, but still heavily biased. Thinking models (o3, o4-mini) significantly outperformed non-thinking models, suggesting explicit reasoning helps detect anomalies.

Task 5: Game Boards

Game boards with modified grid dimensions

Modified Game Boards

Models knew standard dimensions so strongly they couldn't count actual board lines.

Game Boards

Key Finding: Worst overall performance. Models scored 0% on Sudoku and Go boards, confirming fundamental inability to perform basic visual counting in structured settings.

Task 6: Optical Illusions

Original and modified optical illusions

Modified Optical Illusions

VLMs knew illusion patterns but failed when effects were reversed.

Optical Illusions

Task 7: Patterned Grids

Grid patterns with anomalous cells

Anomalous Grid Patterns

Models prioritized pattern completion over visual counting even in novel contexts.

Patterned Grids

Key Finding: Even with novel patterns never seen before, VLMs inferred expected values from surrounding cells rather than counting actual elements in the target cell.

When VLMs make errors, they don't make random mistakes. Instead, 75.70% of all errors are "bias-aligned" - meaning they give the expected answer based on prior knowledge rather than what they actually see in the image.

Bias-aligned errors across domains

Systematic pattern of bias-aligned errors proving models ignore visual evidence

On counterfactual images, VLMs mostly output answers that match biased choices rather than random errors. This systematic pattern proves models actively ignore visual evidence in favor of memorized knowledge.

This is the smoking gun: If models were simply bad at vision, we'd expect random errors. Instead, we see systematic bias toward "correct" textbook answers, proving they're overriding visual information with memorized facts.

We tested five state-of-the-art models. The results are consistently terrible across the board:

Key Finding: 75.70% of all errors were "bias-aligned" - meaning models gave the expected answer based on prior knowledge rather than random mistakes. This proves they're not just bad at vision; they're actively ignoring what they see.

• Medical Imaging: Missing tumors that don't match training patterns.
• Autonomous Vehicles: Failing to see modified road signs.
• Quality Control: Missing defects in manufactured goods.
• Security: Fooled by simple visual modifications.
• Website/App Control: If user interfaces change subtly (buttons, layouts, or icons), biased models may fail to perform tasks correctly, unable to adapt to minor visual modifications.

• False Confidence: Models are wrong but certain.
• Brittleness: Tiny changes cause complete failure.
• Training Flaws: Memorization over understanding.
• Evaluation Gap: Benchmarks miss real-world failure modes.

Current VLMs are sophisticated pattern matching systems, not visual reasoning systems. They excel at recognizing familiar patterns but fail catastrophically when those patterns are even slightly modified.

We tested two approaches to help models perform better. Neither worked significantly:

Prompt: "Please double-check your answer and give your final answer in curly brackets, following the format above."

Improvement: +2.70% (Mean)

Prompt: "Do not assume from prior knowledge and answer only based on what is visible in the image."

Improvement: +1.87% (Mean)

Sobering Reality: Even with explicit instructions to ignore prior knowledge and focus on visual details, models barely improved. The bias is deeply embedded in how they process visual information.

Adding subject names directly to images (like "Ebbinghaus illusion") made models even more biased, dropping accuracy by an additional 4.49%.

Original vs. modified versions without (top) and with (bottom) the in-image text ("Ebbinghaus illusion"). Adding text labels makes models more likely to rely on memorized knowledge rather than visual analysis.

Effect: -4.49% accuracy drop when subject names were added to images.

Worse for thinking models: o4-mini (-6.56), o3 (-6.41) vs. Sonnet-3.7 (-2.81), GPT-4.1 (-2.67).

This suggests that more sophisticated reasoning can sometimes amplify bias when textual cues are present.

The AI community needs to acknowledge that current VLMs have fundamental limitations. We need better evaluation methods that test actual visual reasoning, not just pattern recognition.

• Develop training methods that emphasize visual analysis over memorization.
• Create evaluation benchmarks that test robustness to modifications.
• Build models that can explicitly separate prior knowledge from visual evidence.
• Investigate multi-modal reasoning architectures.

• Implement uncertainty quantification for visual tasks.
• Develop hybrid systems combining vision models with explicit counting modules.
• Create domain-specific fine-tuning approaches.
• Build better human-AI collaboration interfaces.

VLMs aren't as smart as we thought.

They're incredibly sophisticated at recognizing patterns they've seen before, but they fundamentally lack the ability to perform basic visual analysis when faced with novel variations.

This research reveals a critical blind spot in AI development. As we deploy these systems in high-stakes applications, we must understand their limitations. A model that can describe complex scenes but can't count legs on a modified animal is not truly "seeing" - it's performing very sophisticated pattern matching.

"The most dangerous thing about current VLMs isn't that they fail - it's that they fail confidently, giving no indication that they're relying on memorized knowledge rather than actual visual analysis."

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4