Skip to content
GCC AI Research

A new stress test for AI agents that plan, look and click

MBZUAI · Significant research

Research CV NLP LLM Benchmark

Summary

MBZUAI researchers won second place at the AgentX Competition at UC Berkeley for their benchmark measuring AI agents' reasoning across images, comparisons, and video. The Agent-X dataset includes 828 tasks across six domains, requiring agents to use 14 executable tools without explicit instructions. Agent-X analyzes the agent's full reasoning trajectory, unlike typical evaluations that focus only on final answers. Why it matters: The benchmark exposes limitations in current multimodal AI agents and provides a more rigorous evaluation framework for real-world applications in the region and beyond.

Keywords

MBZUAI · AgentX · AI agents · Multimodal reasoning · Benchmark

Read original article →

Get the weekly digest

Top AI stories from the GCC region, every week.

Related

CAPTCHAs aren’t just annoying, they’re a reality check for AI agents

MBZUAI ·

MBZUAI researchers created Open CaptchaWorld, a new benchmark to test AI agents on solving CAPTCHAs. The benchmark includes 20 modern CAPTCHA types that require perception, reasoning, and interactive actions within a browser. While humans achieve 93.3% accuracy, the best AI agent only reaches 40% on the benchmark. Why it matters: This research highlights a critical gap in current AI agent capabilities, as CAPTCHAs are gatekeepers to high-value web actions like e-commerce and secure logins.