InfiAgent is a new agent framework comparable to GPT4-Agent, developed by replicating Codex. It includes InfiCoder, an open-source model for text-to-code, code-to-code, and freeform code-related QA tasks. The framework focuses on data analysis and integrates an LLM with programming capabilities and a sandbox environment for executing Python code. Why it matters: This research demonstrates the potential for advancements in AI operating systems and highlights areas where current models like GPT-4V can be improved, contributing to the broader development of more capable and versatile AI agents.
MBZUAI faculty Eric Xing and Qirong Ho are developing AI operating systems (AI OS) for efficient AI development, similar to mobile OS. They co-founded AI startup Petuum and lead the CASL community, which focuses on composable, automatic, and scalable learning. CASL provides a unified toolkit for distributed training and compositional model construction, with contributions from MBZUAI, CMU, Berkeley, and Stanford. Why it matters: The development of AI OS aims to optimize AI applications by efficiently connecting software and hardware, fostering innovation and broader adoption of AI solutions across industries in the region.
The paper introduces ILION, a deterministic execution gate designed to ensure the safety of autonomous AI agents by classifying proposed actions as either BLOCK or ALLOW. ILION uses a five-component cascade architecture that operates without statistical training, API dependencies, or labeled data. Evaluation against existing text-safety infrastructures demonstrates ILION's superior performance in preventing unauthorized actions, achieving an F1 score of 0.8515 with sub-millisecond latency.
MBZUAI Assistant Professor Qirong Ho is researching AI operating systems to standardize algorithms and enable non-experts to create AI applications reliably. He emphasizes that countries mastering mass production of AI systems will benefit most from the Fourth Industrial Revolution. Ho is co-founder and CTO at Petuum Inc., an AI startup creating standardized building blocks for affordable and scalable AI production. Why it matters: This research aims to democratize AI development and promote widespread adoption across industries in the UAE and beyond.
A new benchmark, LongShOTBench, is introduced for evaluating multimodal reasoning and tool use in long videos, featuring open-ended questions and diagnostic rubrics. The benchmark addresses the limitations of existing datasets by combining temporal length and multimodal richness, using human-validated samples. LongShOTAgent, an agentic system, is also presented for analyzing long videos, with both the benchmark and agent demonstrating the challenges faced by state-of-the-art MLLMs.