Understanding and reasoning about space—like picturing a room from different angles or imagining how objects relate to each other—is something people do naturally. When solving puzzles that require spatial thinking, humans often mentally “step into” different viewpoints to make sense of the scene. But can today’s advanced AI systems, which combine vision and language understanding, do the same? A newly published research paper introduces a novel method called SpatialASM that teaches large vision-language models (LVLMs) to actively imagine and explore spaces from multiple perspectives, improving their ability to solve complex spatial reasoning tasks.
Key Takeaways
- SpatialASM enables AI models to internally simulate viewing a scene from different angles, mimicking how humans actively reconsider spatial layouts.
- The method uses a “novel-view generator” to create hypothetical viewpoints grounded in actual visual data, helping models practice perspective-taking.
- Training with SpatialASM improves performance on seven diverse spatial reasoning benchmarks, from simple scene understanding to complex multi-image and viewpoint transformation tasks.
- This approach enhances spatial reasoning without sacrificing the models’ general vision and language abilities, making it a scalable way to boost AI’s spatial intelligence.
Current large vision-language models have made impressive strides in understanding images and generating descriptions or answers based on visual input. However, they tend to lack a crucial human-like skill: actively imagining how a scene looks from different viewpoints to reason about spatial relationships. This ability, called Active Spatial Mentalization (ASM), involves not just passively observing but dynamically simulating and planning new perspectives to gain better insight.
To teach AI models this skill, the researchers developed SpatialASM, a training framework that encourages LVLMs to engage in spatial simulation during learning. At the heart of this approach is an allocentric novel-view generator—a tool that creates new, hypothetical views of a scene from different angles independent of the model’s current viewpoint. “Allocentric” here means that the viewpoints are defined relative to the environment rather than the observer, allowing the model to imagine the scene as if “moving around” it.
SpatialASM provides two complementary forms of guidance during training. First, it generates explicit ASM trajectories, which are step-by-step paths of imagined viewpoints that teach the model when and how to plan its visual exploration actively. Second, it automatically constructs perspective-taking tasks that challenge the model to interpret scenes conditioned on these new viewpoints. This dual supervision helps the model develop a more flexible and dynamic understanding of spatial layouts.
The researchers tested SpatialASM on seven benchmark datasets covering a range of spatial reasoning challenges. These tasks included straightforward scene perception as well as more demanding problems requiring multi-image analysis, cross-view integration, and viewpoint transformations. Across four different LVLM architectures, SpatialASM consistently improved average accuracy, demonstrating its broad applicability.
Importantly, the improvements in spatial reasoning did not come at the cost of the models’ general visual and language capabilities. This suggests that SpatialASM can be integrated into existing LVLMs to enhance their spatial intelligence without hindering other functions.
Looking ahead, this research points to promising directions for making AI systems better at understanding and interacting with the physical world. Enhanced spatial reasoning could benefit applications like robotics, augmented reality, and navigation assistance, where interpreting complex environments from multiple perspectives is crucial. By internalizing the human-like ability to actively explore and imagine spaces, AI models may become more reliable and versatile partners in tasks requiring spatial awareness.
Based on research published on arXiv by Yuhong Liu, Shenxi Wu, Junyi Chi et al..