π Accepted at ICML 2026
Disentangling Perception, Reasoning, and Simulation in VLM Gameplay
PRS-Diag(Perception, Reasoning, and Simulation Diagnostics) provides two test suites that evaluate VLM capabilities in structured game environments:
-
Perception Tests β Controlled evaluation of visual encoding under variations in density, patch alignment, resolution, and visual richness, plus quantification of spatially coordinated localization drift. Details
-
Rule-Following Tests β A diagnostic matrix and six-level rule complexity ladder evaluated in both explicit verification and predictive simulation modes, with text-only and chain-of-thought ablations. Details
VideoTeaser.mp4
# Developed and tested with Python 3.11.9 # Install rule-following module cd rule_following pip install -e . cd .. # Install perception dependencies cd perception pip install -r requirements.txt cd ..
See individual READMEs for detailed usage.
https://drive.google.com/file/d/1AjAKDChi7B-Ummdt21Vsxq9CdVImxf11/view?usp=sharing
@inproceedings{jin2026seeing, title={Seeing Without Understanding: Disentangling Perception, Reasoning, and Simulation in VLM Gameplay}, author={Jin, Dingyang and He, Jiawei and Lo, Calvin and Hu, Steven and Rad, Ryan}, booktitle={Proceedings of the 43rd International Conference on Machine Learning (ICML)}, year={2026} }