Skip to content
GCC AI Research

CAPTCHAs aren’t just annoying, they’re a reality check for AI agents

MBZUAI · Significant research

Research LLM Robotics Ethics Product

Summary

MBZUAI researchers created Open CaptchaWorld, a new benchmark to test AI agents on solving CAPTCHAs. The benchmark includes 20 modern CAPTCHA types that require perception, reasoning, and interactive actions within a browser. While humans achieve 93.3% accuracy, the best AI agent only reaches 40% on the benchmark. Why it matters: This research highlights a critical gap in current AI agent capabilities, as CAPTCHAs are gatekeepers to high-value web actions like e-commerce and secure logins.

Keywords

CAPTCHA · AI agents · MBZUAI · benchmark · Open CaptchaWorld

Read original article →

Get the weekly digest

Top AI stories from the GCC region, every week.

Related

A new stress test for AI agents that plan, look and click

MBZUAI ·

MBZUAI researchers won second place at the AgentX Competition at UC Berkeley for their benchmark measuring AI agents' reasoning across images, comparisons, and video. The Agent-X dataset includes 828 tasks across six domains, requiring agents to use 14 executable tools without explicit instructions. Agent-X analyzes the agent's full reasoning trajectory, unlike typical evaluations that focus only on final answers. Why it matters: The benchmark exposes limitations in current multimodal AI agents and provides a more rigorous evaluation framework for real-world applications in the region and beyond.