Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

调用vllm接口,最后一个token输出后一直收不到is_end,导致卡十秒钟 #3313

chenslcool started this conversation in General
Discussion options

我有一个api接口,和vllm格式的返回不完全一致,我成功接入了模型,但是调用应用的时候,输出最后一个token后总要卡十秒钟才显示对话结束。接口的格式是这样的,请问这个接口是哪个字段不对呢?导致maxkb不能确认已经停止
image

You must be logged in to vote

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
1 participant

AltStyle によって変換されたページ (->オリジナル) /