Spatial Reasoning for Robots: What ETH Researchers Are Teaching Machines

For us humans, it's second nature: we see a cup, instantly estimate how far away it is, how it's positioned in space, and reach for it. For robots, that's surprisingly difficult. Many modern robot programmes «see» the world primarily as flat camera images – as if they were looking at a photograph. They have to laboriously figure out the third dimension themselves – depth and spatial arrangement. When grasping, where millimetres and actual contact matter, this often leads to errors.

A research team with participation from ETH Zurich – including renowned robotics professors Marco Hutter and Marc Pollefeys – has presented a new approach: the «Geometric Action Model», or GAM for short. The idea is simple to summarize: rather than awkwardly deriving spatial understanding from flat images, GAM uses an AI model from the start that already understands the world in three dimensions.

This spatial foundation serves the robot for three things at once: it perceives its surroundings, it imagines what will happen in the next moment, and it decides how to move. Everything runs through a single «brain» instead of multiple separate systems. Put simply: the robot can spatially visualize its next move before executing it – similar to how a human quickly mentally rehearses reaching for a glass.

According to the study, the results are convincing in multiple ways. In tests – both in simulation and with real robots – GAM worked more precisely and reliably than previous, comparably sized systems. At the same time, it was faster and required less computing power. This is particularly important because robots need to make decisions in real time and cannot carry around unlimited computational resources.

It's still a research project, not a finished product. But it demonstrates a central development direction for the industry: robots should not only see the physical world as an image, but understand it as space. The better they manage this, the sooner they can take on everyday tasks – from organizing a workshop to household chores. The fact that part of this research comes from Zurich once again underscores Switzerland's role as Europe's centre for intelligent robotics.