This repository was archived by the owner on Sep 10, 2025. It is now read-only.
-
Couldn't load subscription status.
- Fork 249
This repository was archived by the owner on Sep 10, 2025. It is now read-only.
torchtune as an optional dependency: Lazy Import #1519
Open
Assignees
@Jack-Khuu
Description
🚀 The feature, motivation and pitch
We would like to make torchtune an optional dependency. The first step towards that is to avoid importing torchtune unless it is actively used.
To make this migration easier, let's move the top level imports into the functions/classes that require them.
We explicitly acknowledge that this initial step isn't a best practice, but will make the following work simpler.
Here's an example where we delay imports here:
torchchat/torchchat/usages/eval.py
Lines 216 to 225 in 1384f7d
# Having the imports here allow running other evals without installing torchtune
from torchtune.utils import batch_to_device
from torchtune.data import (
format_content_with_images,
left_pad_sequence,
Message,
padded_collate_tiled_images_and_mask,
)
from torchtune.generation import generate, sample
from torchtune.modules.common_utils import local_kv_cache
Task: Update all imports of torchtune in the repo, such that imports are only done when necessary
- Suggestion: Feel free to send out PR's that only update a subset of all use cases
To test your changes, run:
- With torchtune installed: https://github.com/pytorch/torchchat/blob/main/docs/multimodal.md#generation
- With torchtune uninstalled:
python torchchat.py generate llama3.2-1B
Alternatives
No response
Additional context
No response
RFC (Optional)
No response
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
Ready