Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings
This repository was archived by the owner on Sep 10, 2025. It is now read-only.
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

What's the future plan for torchchat serving #1491

Open
Assignees
Labels
enhancementNew feature or request triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
@jenniew

Description

I see current torchchat serving provides basic serving function. I'm wondering what the future plan for serving. What's the target of torchchat serve? Will it provide more optimized and high performance serving features(like Continuous batching, prefix-caching, chunked prefill, etc.)

Metadata

Metadata

Labels

enhancementNew feature or request triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

No type

Projects

No projects

Milestone

No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      AltStyle によって変換されたページ (->オリジナル) /