banner

Official implementation of Extendable Crowd-Sourced Splatting (xCSS), an extension to CSS (ICASSP 2025), combining MASt3R’s pose-free matching with NeRFStudio-based Gaussian Splatting.

• 📘 CSS Paper (ICASSP 2025)
• 📘 MASt3R arXiv (2024)
• 📘 DUSt3R arXiv (2023)

xCSS bridges geometry-grounded image matching with 3D Gaussian Splatting to support scene reconstruction using crowd-sourced imagery without requiring known camera poses. By leveraging the MASt3R model for 2D-3D grounding and integrating it with msplat and nerfstudio, this pipeline enables scalable and flexible reconstruction of complex real-world environments.

• xCSS: Extendable Crowd-Sourced Splatting
  • Overview
  • Table of Contents
  • Installation
    • Step 1: Install MASt3R
      • 1. Clone the repository
      • 2. Setup environment with conda
      • 3. Compile and install ASMK (retrieval support)
      • 4. Compile CUDA kernels for RoPE embeddings
      • 5. Download pre-trained MASt3R models
    • Step 2: Install NeRFStudio
    • Step 3: Install msplat
    • Step 4: Run xCSS
  • Citation
  • License

git clone --recursive https://github.com/naver/mast3r
cd mast3r

conda create -n mast3r python=3.11 cmake=3.14.0
conda activate mast3r
conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia
pip install -r requirements.txt
pip install -r dust3r/requirements.txt
pip install -r dust3r/requirements_optional.txt # for HEIC image and visual localization support

pip install cython
git clone https://github.com/jenicek/asmk
cd asmk/cython/
cythonize *.pyx
cd ..
pip install .

cd dust3r/croco/models/curope/
python setup.py build_ext --inplace
cd ../../../../

mkdir -p checkpoints/
wget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric.pth -P checkpoints/
wget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric_retrieval_trainingfree.pth -P checkpoints/
wget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric_retrieval_codebook.pkl -P checkpoints/

Follow the official NeRFStudio installation guide here: 📘 https://docs.nerf.studio/quickstart/installation.html

Ensure it is installed correctly by running:

ns-train --help

We use msplat for the Gaussian Splatting backend:

pip install git+https://github.com/pointrix-project/msplat.git

Once MASt3R, NeRFStudio, and msplat are installed, run the main pipeline using:

bash xcss.sh [-d] [-s SCENE_GRAPH] <root_dir>

Make sure to place your input images in the directory referenced in the xcss.sh script.

This script will:

• Run MASt3R to detect dense matches and compute view correspondence
• Estimate scene structure from unposed images
• Launch NeRFStudio with msplat for pose-free 3D Gaussian Splatting

If you use this project in your research, please cite both CSS and MASt3R:

@INPROCEEDINGS{10889766,
 author={Chen, Runze and Xiao, Mingyu and Luo, Haiyong and Zhao, Fang and Wu, Fan and Xiong, Hao and Liu, Qi and Song, Meng},
 booktitle={ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 
 title={CSS: Overcoming Pose and Scene Challenges in Crowd-Sourced 3D Gaussian Splatting}, 
 year={2025},
 pages={1-5},
 doi={10.1109/ICASSP49660.2025.10889766}
}
@misc{mast3r_arxiv24,
 title={Grounding Image Matching in 3D with MASt3R}, 
 author={Vincent Leroy and Yohann Cabon and Jerome Revaud},
 year={2024},
 eprint={2406.09756},
 archivePrefix={arXiv},
 primaryClass={cs.CV}
}

This project is released under multiple licenses:

• MASt3R: CC BY-NC-SA 4.0 (non-commercial use)
• msplat and NeRFStudio: See their respective repositories
• Please also respect dataset licenses (e.g., MapFree is highly restricted)