IROS 2026 (under review)
Multi-agent reinforcement learning (MARL) commonly relies on a centralized critic to estimate the
value function. However, learning such a critic from scratch is highly sample-inefficient and
often lacks generalization across environments. At the same time, large vision-language-action
models (VLA) trained on internet-scale data exhibit strong multimodal reasoning and zero-shot
generalization capabilities, yet directly deploying them for robotic execution remains
computationally prohibitive, particularly in heterogeneous multi-robot systems with diverse
embodiments and resource constraints. To address these challenges, we propose Multi-Agent
Vision-Language-Critic Models (MA-VLCM), a framework that replaces the learned centralized critic
in MARL with a pretrained vision-language model fine-tuned to evaluate multi-agent behavior.
MA-VLCM acts as a centralized critic conditioned on natural language task descriptions, visual
trajectory observations, and structured multi-agent state information. By eliminating critic
learning during policy optimization, our approach significantly improves sample efficiency while
producing compact execution policies suitable for deployment on resource-constrained robots.
Results show good zero-shot return estimation on models with differing VLM backbones on
in-distribution and out-of-distribution scenarios in multi-agent team settings.