Towards Accurate Instance Segmentation in Large-scale LiDAR Point Clouds

Set up environment

Example 1 of installation

conda create -n treeins_env_local python=3.8
conda activate treeins_env_local
conda install pytorch=1.9.0 torchvision==0.10.0 torchaudio==0.9.0 cudatoolkit=11.1 -c pytorch -c nvidia
pip install numpy==1.19.5
conda install openblas-devel -c anaconda
export CUDA_HOME=/usr/local/cuda-11
pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=${CONDA_PREFIX}/include" –install-option="--blas=openblas"
#CHECK IF TORCH AND MINKOWSKI ENGINE WORK AS EXPECTED:
(treeins_env_local) : python
Python 3.8.13 (default, #DATE#, #TIME#)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True
>>> import MinkowskiEngine
>>> exit()
#CHECK FINISHED
pip install torch-scatter==2.0.8 -f https://data.pyg.org/whl/torch-1.9.0+cu111.html
pip install torch-sparse==0.6.12 -f https://data.pyg.org/whl/torch-1.9.0+cu111.html
pip install torch-geometric==1.7.2
#We got the file requirements.txt from here: https://github.com/nicolas-chaulet/torch-points3d/blob/master/requirements.txt but deleted the lines containing the following libraries in the file: torch, torchvision, torch-geometric, torch-scatter, torch-sparse, numpy
pip install -r requirements.txt
pip install numba==0.55.1
conda install -c conda-forge hdbscan==0.8.27
conda install numpy-base==1.19.2
pip install joblib==1.1.0

Example 2 of installation

conda create -n torchpoint3denv python=3.8
conda activate torchpoint3denv
conda install -c conda-forge gcc==9.4.0
conda install pytorch=1.9.0 torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia
conda install numpy==1.19.5
mamba install libopenblas openblas
find ${CONDA_PREFIX}/include -name "cblas.h"
export CXX=g++
export MAX_JOBS=2;
git clone https://github.com/NVIDIA/MinkowskiEngine.git
cd MinkowskiEngine
python setup.py install --blas_include_dirs=${CONDA_PREFIX}/include --blas=openblas
#THE STEPS START FROM HERE ARE EXACT THE SAME AS EXAMPLE 1 
#CHECK IF TORCH AND MINKOWSKI ENGINE WORK AS EXPECTED: 
#...

Data introduction

NPM3D dataset with instance labels

Semantic labels

Data folder structure

├─ conf # All configurations for training and evaluation leave there
├─ forward_scripts # Script that runs a forward pass on possibly non annotated data
├─ outputs # All outputs from your runs
├─ scripts # Some scripts to help manage the project
├─ torch_points3d
├─ data # DATA FOLDER
 └─ npm3dfused
 └─ raw
 ├─ *_train.ply # All train files
 ├─ *_val.ply # All val files
 └─ *_test.ply # All test files
├─ train.py # Main script to launch a training
└─ eval.py # Eval script

FOR-instance dataset

Different forest regions/subgroups of trees

Train - validation - test split

Data folder structure

├─ conf # All configurations for training and evaluation leave there
├─ forward_scripts # Script that runs a forward pass on possibly non annotated data
├─ outputs # All outputs from your runs
├─ scripts # Some scripts to help manage the project
├─ torch_points3d
├─ data # DATA FOLDER
 └─ treeinsfused
 └─ raw
 ├─ CULS
 ├─ *_train.ply # All train files
 ├─ *_val.ply # All val files
 └─ *_test.ply # All test files
 ├─ NIBIO
 ├─ *_train.ply # SIMILAR AS CULS FOLDER
 ├─ *_val.ply 
 └─ *_test.ply 
 ├─ NIBIO2
 ├─ *.ply # SIMILAR AS CULS FOLDER 
 ├─ RMIT
 ├─ *.ply # SIMILAR AS CULS FOLDER
 ├─ SCION
 ├─ *.ply # SIMILAR AS CULS FOLDER
 └─ TUWIEN
 ├─ *.ply # SIMILAR AS CULS FOLDER
├─ train.py # Main script to launch a training
└─ eval.py # Eval script

Getting started with code

How to set different parameters

# An example for NPM3D dataset
# Run Setting IV for test area 1, radius=16m, voxel side length=0.12m, training for 200 epoches.
python train.py task=panoptic data=panoptic/npm3d-sparseconv_grid_012_R_16_cylinder_area1 models=panoptic/area4_ablation_3heads_5 model_name=PointGroup-PAPER training=7_area1 job_name=A1_S7
# An example for FOR-instance dataset
# Run Setting IV, radius=8m, voxel side length=0.2m, training for 150 epoches.
python train.py task=panoptic data=panoptic/treeins_rad8 models=panoptic/area4_ablation_3heads_5 model_name=PointGroup-PAPER training=treeins job_name=treeins_my_first_run

python eval.py

# For NPM3D dataset
python evaluation_stats_NPM3D.py
# For FOR-instance dataset
python evaluation_stats_FOR.py

Citing

@inproceedings{
 Xiang2023,
 title={Towards Accurate Instance Segmentation in Large-scale LiDAR Point Clouds},
 author={Binbin Xiang and Torben Peters and Theodora Kontogianni and Frawa Vetterli1 and Stefano Puliti and Rasmus Astrup and Konrad Schindler},
 booktitle={2023 The ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
 year={2023},
 url = {\url{https://arxiv.org/abs/2307.02877}}
}