This is the official implementation of the paper "Auto-DAS: Automated Proxy Discovery for Training-free Distillation-aware Architecture Search"
- Python 3.6
- PyTorch 1.4.0
- torchvision 0.5.0
- CUDA 10.1
- apex
- tensorboardX
- tqdm
- numpy
- scipy
- scikit-learn
- matplotlib
Optional:
- NAS-Bench-101: Subset of the dataset with only models trained at 108 epochs:
- NAS-Bench-201:
Download the ImageNet dataset and put it in the folder ./data/imagenet/. The folder structure should be like this:
data
- imagenet
- train
- n01440764
- n01443537
- n01484850
- ...
- val
- n01440764
- n01443537
- n01484850
- ...
Train the teacher model by running the following command:
python exps/train_teacher.py --data_path ./data/imagenet/ --save_path ./exps/teacher/ --arch resnet50 --epochs 90 --batch_size 256 --lr 0.1 --lr_schedule cosine --weight_decay 1e-4 --warmup_epochs 5 --label_smoothing 0.1 --mixup_alpha 0.2 --cutout_size 16 --cutout_prob 1.0 --num_workers 8 --gpu 0
Train the student model by running the following command:
python exps/train_student.py --data_path ./data/imagenet/ --save_path ./exps/student/ --teacher_path ./exps/teacher/ --arch resnet50 --epochs 90 --batch_size 256 --lr 0.1 --lr_schedule cosine --weight_decay 1e-4 --warmup_epochs 5 --label_smoothing 0.1 --mixup_alpha 0.2 --cutout_size 16 --cutout_prob 1.0 --num_workers 8 --gpu 0
Evaluate the student model by running the following command:
python exps/eval_student.py --data_path ./data/imagenet/ --save_path ./exps/student/ --teacher_path ./exps/teacher/ --arch resnet50 --batch_size 256 --num_workers 8 --gpu 0
If you find Auto-DAS useful in your research, please consider citing the following paper:
@inproceedings{sunauto,
title={Auto-DAS: Automated Proxy Discovery for Training-free Distillation-aware Architecture Search},
author={Sun, Haosen and Li, Lujun and Dong, Peijie and Wei, Zimian and Shao, Shitong}
year={2024},
organization={ECCV}
}
This project is licensed under the MIT License.