Team NimbRo's robot Mario won the MBZIRC 2017 Challenge 2 by autonomously manipulating a valve stem using a wrench. The robot uses an omnidirectional base for locomotion and a 3D laser scan detector to find the manipulation panel. A deep neural network detects and selects the correct tool from grayscale images, and motion primitives are adapted to turn the valve stem. Why it matters: This work demonstrates advanced robotic manipulation capabilities relevant for industrial automation and hazardous environment operations in the region.
MBZUAI Professor Sami Haddadin and his team developed a new framework called Tactile Skills to teach robots manual skills through touch and trial and error. This framework aims to address the gap in robots' ability to learn basic physical tasks compared to AI's advancements in language and image generation. The research, published in Nature Machine Intelligence, focuses on enabling robots to perform manipulation skills at industrial levels with low energy and compute demands. Why it matters: This research could lead to robots capable of performing household maintenance, industrial tasks, and even assisting in medical or rehabilitation settings, potentially solving labor shortages in various sectors in the region and beyond.
Lorenzo Jamone from Queen Mary University of London presented on cognitive robotics, focusing on tactile exploration and manipulation by robots. The talk covered combining biology, engineering, and AI for advanced robotic systems. Jamone directs the CRISP group and has over 100 publications in cognitive robotics. Why it matters: This highlights the ongoing research into more sophisticated robotic systems that can interact with complex environments, an area crucial for future applications in manufacturing and human-robot collaboration in the GCC.
Researchers present RUR53, an unmanned ground vehicle (UGV) capable of autonomous navigation, object recognition, and tool manipulation. The UGV uses a modular software architecture, enabling it to perform complex tasks like detecting panels, docking, and manipulating tools such as wrenches and valve stems. RUR53 was tested at the 2017 Mohamed Bin Zayed International Robotics Challenge where it ranked third in the Grand Challenge as part of a collaboration. Why it matters: This research demonstrates advanced robotics capabilities applicable to various industrial and inspection tasks, highlighting the UAE's focus on robotics innovation.
Researchers introduce MATRIX, a vision-centric agent tuning framework for robust tool-use reasoning in VLMs. The framework includes M-TRACE, a dataset of 28.5K multimodal tasks with 177K verified trajectories, and Pref-X, a set of 11K automatically generated preference pairs. Experiments show MATRIX consistently outperforms open- and closed-source VLMs across three benchmarks.