This is the official repository for the paper:

Code2Worlds: Empowering Coding LLMs for 4D World Generation

Yi Zhang*, Yunshuang Wang*, Zeyu Zhang*^†, and Hao Tang^‡

School of Computer Science, Peking University

*Equal contribution. ^†Project lead. ^‡Corresponding author

Paper | Website | Code4D | HF Paper

If you find our code or paper helpful, please consider starring ⭐ us and citing:

@article{zhang2026code2worlds,
 title={Code2Worlds: Empowering Coding LLMs for 4D World Generation},
 author={Zhang, Yi and Wang, Yunshuang and Zhang, Zeyu and Tang, Hao},
 journal={arXiv preprint arXiv:2602.11757},
 year={2026}
}

Achieving spatial intelligence requires moving beyond visual plausibility to build world simulators grounded in physical laws. While coding LLMs have advanced static 3D scene generation, extending this paradigm to 4D dynamics remains a critical frontier. This task presents two fundamental challenges: multi-scale context entanglement, where monolithic generation fails to balance local object structures with global environmental layouts; and a semantic-physical execution gap, where open-loop code generation leads to physical hallucinations lacking dynamic fidelity. We introduce Code2Worlds, a framework that formulates 4D generation as language-to-simulation code generation. First, we propose a dual-stream architecture that disentangles retrieval-augmented object generation from hierarchical environmental orchestration. Second, to ensure dynamic fidelity, we establish a physics-aware closed-loop mechanism in which a Post-Process Agent scripts dynamics, coupled with a VLM-Motion Critic that performs self-reflection to iteratively refine simulation code.Evaluations on the Code4D benchmark show Code2Worlds outperforms baselines with a 41% SGS gain and 49% higher Richness, while uniquely generating physics-aware dynamics absent in prior static methods.

image

2026年02月15日: 🎉 Our paper has been promoted by CVer.

• Upload our paper to arXiv and build project pages.
• Add a demo.
• Upload the code.
• Upload the Code4D

1. Install dependencies:

pip install -r requirements.txt

1. Install Infinigen from the official repository:

git clone https://github.com/princeton-vl/infinigen.git
cd infinigen
pip install -e .

For detailed Infinigen installation instructions, please refer to the official documentation.

Star History Chart

We thank the authors of Infinigen for their open-source code.