-
Notifications
You must be signed in to change notification settings - Fork 4.3k
How do I stream chat completions with OpenAI’s Python API? #2462
-
I'm using the official openai Python package to call the Chat API (gpt-3.5 or gpt-4), and I'd like to stream the response instead of waiting for the full reply.
I tried this:
response = openai.ChatCompletion.create( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello"}] )
But it waits until everything is returned.
How can I make it stream the tokens one by one as they’re generated?
Beta Was this translation helpful? Give feedback.
All reactions
Great question!
To stream chat completions using the openai Python package, you need to set stream=True and then iterate over the events.
Here’s how you can do it:
import openai openai.api_key = "your-api-key" response = openai.ChatCompletion.create( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}], stream=True # ✅ this enables streaming ) for chunk in response: if "choices" in chunk: content = chunk["choices"][0]["delta"].get("content", "") print(content, end="", flush=True)
This will print the generated message token-by-token in real time.
Let me know if that works — and feel free to mark this as the answer if it helps! ✅
Replies: 2 comments 2 replies
-
Great question!
To stream chat completions using the openai Python package, you need to set stream=True and then iterate over the events.
Here’s how you can do it:
import openai openai.api_key = "your-api-key" response = openai.ChatCompletion.create( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}], stream=True # ✅ this enables streaming ) for chunk in response: if "choices" in chunk: content = chunk["choices"][0]["delta"].get("content", "") print(content, end="", flush=True)
This will print the generated message token-by-token in real time.
Let me know if that works — and feel free to mark this as the answer if it helps! ✅
Beta Was this translation helpful? Give feedback.
All reactions
-
Thank you so much!
Beta Was this translation helpful? Give feedback.
All reactions
-
here is a detailed explanation : https://cookbook.openai.com/examples/how_to_stream_completions
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
@AhmedGMurtaza Thanks for the link to the article, I didn't think OpenAI had released an official paper on this specific topic!
In any case I will gladly read it to find out more
Beta Was this translation helpful? Give feedback.