InclusionAI Deploys VISTA-9B To Map Text Commands To Screen Clicks
VISTA-9B is a visual model that understands screen layouts and translates natural language instructions into precise click coordinates. It looks at a screenshot and figures out exactly where to click based on what you ask it to do. The system normalizes these coordinates on a zero to one thousand scale to ensure accurate targeting.
The team at inclusionAI who are also behind the Ling-2.6-flash model created this tool by training it on the Qwen3.5 9B foundation. They developed a new training approach called view-consistent self-verified training to help the model handle different screen sizes and layouts. This method exposes the system to semantically similar but geometrically varied screenshots during the learning process.
Model features and target users
Key Features
- Maps natural instructions to click coordinates.
- Trained on Qwen3.5 nine billion backbone.
- Handles geometrically different cropped screenshot views.
- Uses deterministic decoding for accurate results.
People who build automated desktop agents will find this release highly useful for interfacing with visual applications. It allows systems to interact with software interfaces that lack traditional code hooks. Anyone needing to automate user interface interactions can rely on this model for precise visual navigation.
Development details and training requirements
Developers recommend having at least eight high memory GPUs to handle the training process for this architecture. The model avoids blindly copying failed coordinate generations by only adding oracle coordinates when a successful prediction occurs. Testing shows it outperforms standard GRPO methods by about one point on the ScreenSpot Pro benchmark.
"VISTA-9B is a GUI-grounding model that maps a screenshot and a natural-language instruction to a click coordinate in the normalized 0-1000 image frame." - Source: Hugging Face
Project Links
