GitHub - Robotics-STAR-Lab/ApexNav: [RA-L'25 & ICRA'26] An Reliable and Efficient Framework for Zero-Shot Object Navigation

An Adaptive Exploration Strategy for Zero-Shot Object Navigation with Target-centric Semantic Fusion

IEEE Robotics and Automation Letters && IEEE ICRA 2026
Mingjie Zhang ^{1, 2}, Yuheng Du¹, Chengkai Wu ¹, Jinni Zhou¹, Zhenchao Qi¹, Jun Ma ¹, Boyu Zhou ^2,†

¹ The Hong Kong University of Science and Technology (Guangzhou).
² Southern University of Science and Technology.
^†Corresponding Authors

Paper Paper Project Page Lite Version

apexnav_demo
ApexNav ensures highly reliable object navigation by leveraging Target-centric Semantic Fusion, and boosts efficiency with its Adaptive Exploration Strategy.

📢 News

[27/04/2026]: 🌟 A lightweight version is now available! Thanks to TracyLucia for contributing the lightweight version. Check out the Lite-Apexnav branch.
[10/02/2026]: ROS2 Jazzy support is now available! Thanks to romaster93 for contributing the ROS2 interface. Check out the ros2-jazzy branch.
[10/12/2025]: ApexNav released real world test example code. Check out the Real World README for more details.
[07/09/2025]: ApexNav has been published in the Early Access area on IEEE Xplore.
[22/08/2025]: Release the main algorithm of ApexNav.
[18/08/2025]: ApexNav is conditionally accepted to RA-L 2025.

📜 Introduction

[RA-L'25] This repository maintains the implementation of "ApexNav: An Adaptive Exploration Strategy for Zero-Shot Object Navigation with Target-centric Semantic Fusion".

The pipeline of ApexNav is detailed in the overview below.

pipeline

Please kindly star ⭐ this project if it helps you. Thanks for your support! 💖

🛠️ Installation

Tested on Ubuntu 20.04 with ROS Noetic and Python 3.9

You need to install ROS, and it is recommended to use Anaconda or Miniconda to manage your Python environment.

1. Prerequisites

1.1 System Dependencies

sudo apt update
sudo apt-get install libarmadillo-dev libompl-dev
# OSQP
git clone --recursive -b v0.6.3 https://github.com/osqp/osqp.git
cd osqp && mkdir build && cd build
cmake .. -DBUILD_SHARED_LIBS=ON && make -j && sudo make install
cd ../..
# OSQP-Eigen
git clone -b v0.8.1 https://github.com/robotology/osqp-eigen.git
cd osqp-eigen && mkdir build && cd build
cmake .. && make -j && sudo make install
cd ../..

1.2 LLM (Optional)

You can skip LLM configuration and directly use our pre-generated LLM output results in llm/answers.

ollama

curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen3:8b

1.3 External Code Dependencies

git clone git@github.com:WongKinYiu/yolov7.git # yolov7
git clone https://github.com/IDEA-Research/GroundingDINO.git # GroundingDINO

1.4 Model Weights Download

Download the following model weights and place them in the data/ directory:

mobile_sam.pt: https://github.com/ChaoningZhang/MobileSAM/tree/master/weights/mobile_sam.pt

groundingdino_swint_ogc.pth:

wget -O data/groundingdino_swint_ogc.pth https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth

yolov7-e6e.pt:

wget -O data/yolov7-e6e.pt https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7-e6e.pt

2. Setup Python Environment

2.1 Clone Repository

git clone git@github.com:Robotics-STAR-Lab/ApexNav.git
cd ApexNav

2.2 Create Conda Environment

conda env create -f apexnav_environment.yaml -y
conda activate apexnav

2.3 Pytorch

# You can use 'nvcc --version' to check your CUDA version.
# CUDA 11.8
pip install torch==2.5.0 torchvision==0.20.0 torchaudio==2.5.0 --index-url https://download.pytorch.org/whl/cu118
# CUDA 12.1
pip install torch==2.5.0 torchvision==0.20.0 torchaudio==2.5.0 --index-url https://download.pytorch.org/whl/cu121
# CUDA 12.4
pip install torch==2.5.0 torchvision==0.20.0 torchaudio==2.5.0 --index-url https://download.pytorch.org/whl/cu124

2.4 Habitat Simulator

We recommend using habitat-lab v0.3.1

# habitat-lab v0.3.1
git clone https://github.com/facebookresearch/habitat-lab.git
cd habitat-lab; git checkout tags/v0.3.1;
pip install -e habitat-lab
# habitat-baselines v0.3.1
pip install -e habitat-baselines

Note: Any numpy-related errors will not affect subsequent operations, as long as numpy==1.23.5 and numba==0.60.0 are correctly installed.

2.5 Others

pip install salesforce-lavis==1.0.2 # -i https://pypi.tuna.tsinghua.edu.cn/simple
cd .. # Return to ApexNav directory
pip install -e .

Note: Any numpy-related errors will not affect subsequent operations, as long as numpy==1.23.5 and numba==0.60.0 are correctly installed.

📥 Datasets Download

Official Reference: https://github.com/facebookresearch/habitat-lab/blob/main/DATASETS.md

🏠 Scene Datasets

Note: Both HM3D and MP3D scene datasets require applying for official permission first. You can refer to my commands below, and if you encounter any issues, please refer to the official documentation at https://github.com/facebookresearch/habitat-lab/blob/main/DATASETS.md.

HM3D Scene Dataset

Apply for permission at https://matterport.com/habitat-matterport-3d-research-dataset.
Download https://api.matterport.com/resources/habitat/hm3d-val-habitat-v0.2.tar.
Save hm3d-val-habitat-v0.2.tar to the ApexNav/ directory, and the following commands will help you extract and place it in the correct location:

mkdir -p data/scene_datasets/hm3d/val
mv hm3d-val-habitat-v0.2.tar data/scene_datasets/hm3d/val/
cd data/scene_datasets/hm3d/val
tar -xvf hm3d-val-habitat-v0.2.tar
rm hm3d-val-habitat-v0.2.tar
cd ../..
ln -s hm3d hm3d_v0.2 # Create a symbolic link for hm3d_v0.2

MP3D Scene Dataset

Apply for download access at https://niessner.github.io/Matterport/.
After successful application, you will receive a download_mp.py script, which should be run with python2.7 to download the dataset.
After downloading, place the files in ApexNav/data/scene_datasets.

🎯 Task Datasets

# Create necessary directory structure
mkdir -p data/datasets/objectnav/hm3d
mkdir -p data/datasets/objectnav/mp3d
# HM3D-v0.1
wget -O data/datasets/objectnav/hm3d/v1.zip https://dl.fbaipublicfiles.com/habitat/data/datasets/objectnav/hm3d/v1/objectnav_hm3d_v1.zip
unzip data/datasets/objectnav/hm3d/v1.zip -d data/datasets/objectnav/hm3d && mv data/datasets/objectnav/hm3d/objectnav_hm3d_v1 data/datasets/objectnav/hm3d/v1 && rm data/datasets/objectnav/hm3d/v1.zip
# HM3D-v0.2
wget -O data/datasets/objectnav/hm3d/v2.zip https://dl.fbaipublicfiles.com/habitat/data/datasets/objectnav/hm3d/v2/objectnav_hm3d_v2.zip
unzip data/datasets/objectnav/hm3d/v2.zip -d data/datasets/objectnav/hm3d && mv data/datasets/objectnav/hm3d/objectnav_hm3d_v2 data/datasets/objectnav/hm3d/v2 && rm data/datasets/objectnav/hm3d/v2.zip
# MP3D
wget -O data/datasets/objectnav/mp3d/v1.zip https://dl.fbaipublicfiles.com/habitat/data/datasets/objectnav/m3d/v1/objectnav_mp3d_v1.zip
unzip data/datasets/objectnav/mp3d/v1.zip -d data/datasets/objectnav/mp3d/v1 && rm data/datasets/objectnav/mp3d/v1.zip

Make sure that the folder `data` structure has the following structure:

data
├── datasets
│  └── objectnav
│  ├── hm3d
│  │  ├── v1
│  │  │  ├── train
│  │  │  ├── val
│  │  │  └── val_mini
│  │  └── v2
│  │  ├── train
│  │  ├── val
│  │  └── val_mini
│  └── mp3d
│  └── v1
│  ├── train
│  ├── val
│  └── val_mini
├── scene_datasets
│  ├── hm3d
│  │  └── val
│  │  ├── 00800-TEEsavR23oF
│  │  ├── 00801-HaxA7YrQdEC
│  │  ├── .....
│  ├── hm3d_v0.2 -> hm3d
│  └── mp3d
│  ├── 17DRP5sb8fy
│  ├── 1LXtFkjw3qL
│  ├── .....
├── groundingdino_swint_ogc.pth
├── mobile_sam.pt
└── yolov7-e6e.pt

Note that train and val_mini are not required and you can choose to delete them.

🚀 Usage

All following commands should be run in the apexnav conda environment

ROS Compilation

catkin_make -DPYTHON_EXECUTABLE=/usr/bin/python3

Run VLMs Servers

Each command should be run in a separate terminal.

python -m vlm.detector.grounding_dino --port 12181
python -m vlm.itm.blip2itm --port 12182
python -m vlm.segmentor.sam --port 12183
python -m vlm.detector.yolov7 --port 12184

Launch Visualization and Main Algorithm

source ./devel/setup.bash && roslaunch exploration_manager rviz.launch # RViz visualization
source ./devel/setup.bash && roslaunch exploration_manager exploration.launch # ApexNav main algorithm

📊 Evaluate Datasets in Habitat

You can evaluate on all episodes of a dataset.

# Need to source the workspace
source ./devel/setup.bash
# Choose one datasets to evaluate
python habitat_evaluation.py --dataset hm3dv1
python habitat_evaluation.py --dataset hm3dv2 # default
python habitat_evaluation.py --dataset mp3d
# You can also evaluate on one specific episode.
python habitat_evaluation.py --dataset hm3dv2 test_epi_num=10 # episode_id 10

If you want to generate evaluation videos for each episode (videos will be categorized by task results), you can use the following command:

python habitat_evaluation.py --dataset hm3dv2 need_video=true

🎮 Keyboard Control in Habitat

You can also choose to manually control the agent in the Habitat simulator:

# Need to source the workspace
source ./devel/setup.bash
python habitat_manual_control.py --dataset hm3dv1 # Default episode_id = 0
python habitat_manual_control.py --dataset hm3dv1 test_epi_num=10 # episode_id = 10

🤖 Real-world Deployment Example

If you want to run the real-world test example inside the Habitat simulator, please refer to the Real World README for more details.

apexnav_demo2
Trajectory Planning and MPC Control in Real-world Deployment Example in Habitat Simulator.

📋 TODO List

Release the main algorithm of ApexNav
Complete Installation and Usage documentation
Add datasets download documentation
Release the code of real-world deployment
Add ROS2 support
Add the lightweight version of ApexNav

📚 Acknowledgment

We would like to acknowledge the contributions of the following projects:

VLFM : For the concept of Vision-Language Frontier Maps.
FUEL : For the TSP-based efficient frontier exploration framework.

✒️ Citation

@ARTICLE{zhang2025apexnav,
 author={Zhang, Mingjie and Du, Yuheng and Wu, Chengkai and Zhou, Jinni and Qi, Zhenchao and Ma, Jun and Zhou, Boyu},
 journal={IEEE Robotics and Automation Letters}, 
 title={ApexNAV: An Adaptive Exploration Strategy for Zero-Shot Object Navigation With Target-Centric Semantic Fusion}, 
 year={2025},
 volume={10},
 number={11},
 pages={11530-11537},
 keywords={Semantics;Navigation;Training;Robustness;Detectors;Noise measurement;Geometry;Three-dimensional displays;Object recognition;Faces;Search and rescue robots;vision-based navigation;autonomous agents},
 doi={10.1109/LRA.2025.3606388}}

Name		Name	Last commit message	Last commit date
assets		assets
basic_utils		basic_utils
config		config
habitat2ros		habitat2ros
llm		llm
real_world_test_example		real_world_test_example
src		src
vlm		vlm
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
apexnav_environment.yaml		apexnav_environment.yaml
habitat_evaluation.py		habitat_evaluation.py
habitat_manual_control.py		habitat_manual_control.py
habitat_vel_control.py		habitat_vel_control.py
params.py		params.py
pyproject.toml		pyproject.toml

Uh oh!

Latest commit

History

Folders and files

Repository files navigation