-
Notifications
You must be signed in to change notification settings - Fork 2.4k
调用vllm接口,最后一个token输出后一直收不到is_end,导致卡十秒钟 #3313
chenslcool
started this conversation in
General
-
我有一个api接口,和vllm格式的返回不完全一致,我成功接入了模型,但是调用应用的时候,输出最后一个token后总要卡十秒钟才显示对话结束。接口的格式是这样的,请问这个接口是哪个字段不对呢?导致maxkb不能确认已经停止
image
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment